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In accordance with Engrossed Substitute Senate Bill 6168 Section 225(3), this report is in response to the 
Legislature’s request for an evaluation of the Department of Children, Youth and Families Integrated 
Treatment Model. 


(m) $200,000 of the general fund—state appropriation for fiscal year 2020 is provided for the 
department to measure the fidelity of the evidence-based interventions incorporated into the 
integrated treatment model. By July 1, 2020, the department must report to the Governor and the 
appropriate fiscal and policy committees of the Legislature on the results of the assessment of 

the integrated treatment model. 


About Juvenile Rehabilitation 

Juvenile Rehabilitation (JR) became part of Washington State’s Department of Children, Youth, and Families 
(DCYF) on July 1, 2019. JR serves the state’s highest-risk youth who have been charged with a qualifying 
offense and either adjudicated in a county’s juvenile court or convicted in an adult criminal court. As the state 
moves away from institutionalizing youth and the youth crime rate declines, JR has gone from an average daily 
population in 1998 of 1,272, to 402 in 2019, an almost 68% decline in 20 years. Many of the youth who have 
historically been served at JR are now receiving local sanctions. This change has resulted in a changing profile 
of youth (i.e. higher proportion of high-risk youth) committed to JR. As of 2019, JR runs three institutions and 
eight community facilities. Youth start at an institution, then, depending on risk level, sentence, suitability and 
bed availability, can be moved to a community facility before being released. About half of JR youth receive 
community supervision (parole) when they are released from residential care. 


The Integrated Treatment Model 

In 19991, JR implemented a competency-based treatment and case management model. The model focused 
on increasing youth accountability, skill development and measuring youth changes in skill areas throughout 
their stay in the JR continuum of care. 


In 2000, JR needed to further define and specify the appropriate interventions with both the individual youth 
in residential care and subsequently in families as the youth returned to their home communities. JR formed 
the Integrated Treatment Model (ITM) Workgroup, charged with the task of developing a research-based 
treatment model that utilized cognitive-behavioral principles. The model was to be tailored for use in both 
residential and community settings in the juvenile justice continuum of care. Goals for the model included: 


e Research-based effectiveness 

e Motivation and engagement of both youth and families 

e Acommonly understood language to be utilized throughout the juvenile justice continuum 

e Auniform set of cognitive-behavioral skills 

e The ability to generalize and maintain positive changes 

e Ongoing clinical consultation system to ensure the continuity of the interventions and adherence 
to the model 


The workgroup’s report was finalized in 2002 and the process of implementing the ITM began. This model 
views all behavior, including a youth’s criminal behavior as occurring in a larger social and historical context, 


1 This section is adapted from the Executive Summary of the original ITM Report from 2002. 
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serving a specific function. As such, criminal behavior is a product of one’s learning history, encompassing 
family dynamics, specific circumstances and thoughts and feelings. 


For example, in residential care (state institutions and community facilities) the model focuses on improving 
the skills of the youth who are separated from their family and removed from the community context in which 
their behavior occurred. Assessment of the criminal and other undesirable behavior uses a behavioral analysis 
to identify the contextual variables and the function of the behavior. Using basic behavioral change techniques 
of shaping, reinforcement, extinction and contingency management, the therapist implementing the model 
engages the youth in the change process, targets behavior using a hierarchy system and then teaches the 
youth specific behavioral skills to change their actions, thoughts or feelings. Much of the theoretical basis of 
the residential treatment component of the ITM is based on the researched-based treatments of Cognitive 
Behavioral Therapy (Linehan, 1993), Aggression Replacement Training (Glick, Goldstein, and Gibbs, 1998), and 
Moral Reconation Therapy (Little and Robinson, 1988), although the latter has never been fully implemented. 


Once a youth leaves residential care and moves back into the community, the context of their behavior 
changes. The ITM reflects this difference. In community settings, where youth are monitored while on parole, 
the primary focus shifts to creating a more functional environment within the family where the youth resides. 
Again, research on maintaining and supporting behavior change for at-risk adolescents indicates intervention 
is most effective if supported within a family context. Parole staff work with families to shift the “problem 
behavior” to a relational issue between family members. The primary theoretical underpinnings for this 
section of the model come from Functional Family Therapy (Alexander and Parsons, 1982; Sexton and Turner, 
2010), a research-based family intervention. At the onset of the ITM (early 2000s), all youth received some 
level of parole supervision; at the time of this report (2020), only about 50% of youth leaving a JR residential 
facility receive some form of community supervision. 


Since its original inception in 2002, the ITM has evolved to include a risk and needs assessment called the 
Integrated Treatment Assessment (ITA) as well as other assessments that help determine placement, 
treatment needs, and facility and parole eligibility. The main residential treatment model is Dialectical 
Behavior Therapy (DBT) and the main parole treatment model is Functional Family Parole (FFP) with 
Functional Family Therapy (FFT) for those with high needs. Aggression Replacement Training (ART) is another 
treatment offered to eligible youth in the institutions. Additionally, there are Specialized Sex Offender 
Treatment (SOT), Substance Use Disorder (SUD) treatment and Mental Health (MH) treatment available in the 
institutions. Rounding out the ITM, there are protocols in place for suicide and self-harm prevention. The goal 
is this array of assessments and treatments are integrated and aligned using a Risk-Need-Responsivity (RNR) 
framework. The RNR freamework is comprised of three principles. First the “risk principle,” which suggests 
that those with the highest risk for reoffending should be prioritized for treatments and other interventions. 
Second, the “need principle” recommends that the individual needs of each youth are determined, specifically 
those needs that are most likely to be associated with criminal behavior. Third, the “responsivity principle” 
requires that the correct type of programming be offered based on an individual’s risk and need profile (Crites 
and Taxman, 2013; Brogan, Haney-Caron, NeMoyer, and DeMatteo, 2015). 


In 2009, a study was conducted by The Washington State Department of Social and Health Services, Research 
and Data Analysis Division (Lucenko and Mancuso, 2009), that compared outcomes for JR youth from before 
and after the ITM was implemented. Allowing a two-year period to fully implement the ITM, the study 
compared state fiscal year 2002 outcomes to 2006, for youth who were released from JR. The study found 
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that employment rates for JR youth increased by 34% and re-arrest rates declined by 10%, coinciding with the 
implementation of the ITM. While parts of the ITM have been evaluated for their effectiveness over the years, 
there has yet to be an assessment of the implementation of the ITM as a comprehensive treatment model. 


Starting in early 2016, an internal JR committee (called the ITM Reboot Committee) convened to start 
integrating a Positive Youth Development (PYD) framework into the ITM and to examine ITM implementation 
issues. The three main priorities identified by the committee were to improve and streamline training, clarify 
supervision standards, and update quality assurance tools. The committee finalized its findings in early 2018 
and JR leadership has been working since then to address the issues raised by the committee. 


Current Study 

In 2019, the Washington State Legislature allocated funds for a fidelity assessment of the ITM in DCYF’s 
Juvenile Rehabilitation. To accomplish this task, the DCYF Office of Innovation, Alignment and Accountability 
(OIAA) collaborated with external experts to assess the current implementation fidelity of the treatment areas 
in the ITM. The following treatment areas of the ITM were identified as part of the current assessment: 


Assessment System 

Dialectical Behavior Therapy (DBT) 

Functional Family Parole (FFP)/Functional Family Therapy (FFT) 
Aggression Replacement Training (ART) 

Specialized Sex offender Treatment (SOT) 

Substance Use Disorder Treatment (SUD) 

Specialized Mental Health Treatment (MH) 

Suicide and Self-Harm Prevention (SSP) 


CS Oy OB Ne 


For the assessment system, the consultants were asked to address questions related to the appropriateness, 
quality and monitoring of the assessments used to inform the ITM. The specific questions can be found in 
Appendix A. For all the treatment areas, the consultants were asked to address questions related to treatment 
quality, integration and monitoring. The specific questions can be found in Appendix B. The consultants 
reviewed available policies and documentation, studied the available published literature and conducted site 
visits at JR facilities and parole offices before providing their expert responses to the questions. All the original 
consultant reports can be found in the Appendix C through J. Below, we summarize the findings across all the 
treatment areas and provide recommendations for JR’s ITM moving forward. 


Main Findings on Current ITM Implemetation 

Here we present first the findings that apply to the ITM as a whole, and then we summarize the area-specific 
findings. Overall, there are four main findings from this study. First, consultants consistently reported that JR 
has passionate and motivated staff, however, in many cases they are being asked to work outside of their job 
description or are not adequately trained for the job they are being asked to complete. Second, the JR 
organizational structure often impedes effective treatment integration and delivery. Third, the assessment 
system lacks oversight and has not been effectively integrated into treatment eligibility and dosage decisions. 
Fourth, there are inconsistent quality assurance and implementation monitoring practices across JR’s 
integrated treatment model. 
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Finding One: Staff Are Passionate, but Not Adequately Trained 

Across treatment areas, consultants noted the dedication and compassion of JR staff. The interaction between 
youth and front-line staff is the key to effective implementation of the ITM. Having dedicated staff is essential 
to provide a quality treatment model. Dedicated staff, however, are not sufficient. Nearly all the consultants 
mentioned staffing issues, including high turnover, low pay, staffing levels and lack of training. Staff must be 
supported and adequately trained for the treatment to produce the desired outcomes. Importantly, JR 
recently completed a staffing study for both the residential facilities and parole services. The findings from 
those two studies are relevant here and confirm the findings from the consultants in this study. 


Some of the relevant findings from the residential staffing model study include: (1) “The current number of 
staff and composition of the workforce cannot support full implementation of all the components of the 
Rehabilitation Model,” (2) “Staff do not receive the necessary training time to gain requisite knowledge and 
skills for working with youth. In addition, training is not always provided in a timely manner.” (3) “Actual 
turnover and turnover intent of staff are significant issues.” (4) Line-level staff “all appear to be ‘under 
classified’ and underpaid relative to other positions with similar qualifications and responsibilities” (Hyzer 
Group, 2019a). Additionally, the parole services staffing model studies concluded that the current model 
“does not account for the time to conduct the range of necessary reentry services youth require in order to 
successfully reenter their homes and communities. The current model also does not account for the time 
parole counselors spend with youth who are still in residence” (Hyzer Group, 2019b). 


Taken together, from the current report and the extensive staffing model studies that were completed last 
year, it is clear that the supports (number of staff, staff pay and training) do not match the expectations 
required of the treatment model. This imbalance is likely driving the high staff turnover rates, which 
exacerbates the staffing challenges. 


Finding Two: JR Organizational Structure Impedes Effective Treatment 

Juvenile Rehabilitation offers an array of treatment options, and currently most of the important treatment 
areas for effective rehabilitation are present, at some level. However, there are a number of occasions where 
it is evident that the organizational structure is inhibiting effective integration of treatment. Currently, there is 
no clear oversight structure for the Integrated Treatment Model. Treatment administrators are spread 
throughout the organization and do not have direct authority over how treatment is administered. In short, 
the ITM has never been fully integrated, and the agency is not attending to the on-going process of 
integration. 


The agency is teaching youth many skills through different treatments. For example, DBT, ART and FFP all have 
a full set of skills and many youth will be exposed to all three sets of skills. While there are commonalities 
across these treatments, each contains distinct elements and there is little to no attempt to help youth 
understand how all these sets of skills are related. This is largely due to the fact that staff have not been 
trained on all the sets of skills being presented throughout the ITM. Youth will not be able to generalize the 
skills they have learned — the therapeutic step where the youth implements these skills in their own life — if 
staff are not trained to reinforce the skills while they are in JR. 
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Finding Three: Assessment System Lacks Oversight and Is Not Integrated Into Decisions 

The JR assessment system is currently using assessment instruments that have not been validated. 
Additionally, there is no oversight of the assessments being used to make important treatment and residential 
housing decisions. This lack of oversight could result in differential assessment results and differential access 
to services depending on who conducted the assessment. Finally, the assessments are generally not 
integrated into treatment decision-making (i.e. the assessment does not directly inform the type of treatment 
needed or dosage of treatment). Too often treatment is being determined by which facility a youth is placed 
at, and the interventions available at that location, instead of the RNR principles. 


Finding Four: Inconsistent Monitoring of Treatment Quality 

There is a patchwork of quality assurance practices across JR’s ITM. In some treatment areas, there are full- 
time staff who conduct observations and provide a highly reliable assessment of the residential environment 
(for example DBT environmental adherence assessments), while in other areas, very little quality assurance 
has been implemented (substance abuse treatment, see Appendix H and other DBT treatment modes, see 
Appendix D). Additionally, for most of the treatment areas, the treatment administrator is also responsible for 
providing the quality assurance monitoring. This creates a situation where the treatment administrators are 
being asked to provide objective monitoring of their own program, which is contrary to best practice in quality 
assurance. The result is inconsistent, and in many areas, inadequate, quality assurance protocols across the 
ITM. 


Treatment Specific Findings 


Summary of Assessment Findings 

The ITM relies on one risk and needs assessment, called the Integrated Treatment Assessment (ITA) and three 
actuarial risk assessments, the Risk Assessment-Recidivism (RAR), Risk Assessment-Institution (RAI) and Risk 
Assessment-Community Facility (RACF). These four assessments are not subjected to oversight and do not 
have a governance structure. None of them have been validated locally using a JR sample. 


“Of the four instruments the ITA holds the greatest potential to drive case planning, youth classification (on 
risk and need), treatment models and the assessment of change over time. While the ITA does provide a great 
deal of important information, it does not appear that it is currently integrated into the treatment model as 
much as it could be. For example, treatment activities are driven largely by [living unit] placement, which 
appears to be driven by procedures that do not involve the ITA to a substantial degree. Overall, the potential 
for the ITA is not currently being realized” (Holsinger and Holsinger, 2020 — see Appendix C for the full 
assessment system report). 


Summary of Dialectical Behavior Therapy (DBT) Findings 


“The intention [of JR] to provide DBT to fidelity is deeply rooted — from the behaviorally specific DBT standards 
for each DBT mode to the sophisticated and rich intranet of DBT resources to aid in its delivery. Staff shortages 
and staff turn-over have significantly compromised JR’s ability to train up its workforce in DBT and to maintain 


? The only evidence that the items in the assessments are associated with the outcomes come from the construction sample, from 
when the assessments were created. In order for the instruments to be considered validated, the instruments need to be tested 
using a sample that is different from the one used to create the assessments. 
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whatever training gains it makes. Non-competitive wages make it particularly challenging to recruit personnel 
at all levels (AA, BA, MA, PhD). The situation is even more dire in rural areas. 


Training, consultation and supervision resources are insufficient to meet the actual demand that counselors 
have in order to fulfill the DBT standards and deliver DBT to fidelity. In contrast to the early years of DBT 
implementation at JR where staff attended intensive trainings with ample consultation from other JR DBT 
experts, the trainings are now limited to two and three-day trainings and very limited consultation. Staff 
shortages have made it difficult for new staff to attend the trainings that are offered.” 


“The DBT standards accurately capture what a gold-standard JR system should seek to do... With respect to 
the individual delivery of DBT... the majority of today’s counselors have not had enough training to know how 
to deliver DBT individual counseling to fidelity” (Dimeff, 2020a — see Appendix D for the full DBT report). 


Summary of Functional Family Parole (FFP) Findings 

Currently only about half of youth released from JR residential facilities receive some level of aftercare/parole 
services. About half who receive parole do so because they committed a qualifying offense, the other half are 
eligible because they score in the highest 25% on a JR risk assessment. The risk assessment used to determine 
parole eligibility has not been validated. “Without validation, the accuracy of placements is unknown and 
significant numbers of youth may be underserved due to scores that result in no JR parole placement upon 
reentry into their communities, while many other youths at lower risk may be placed” on parole. “It is likely 
that a significant number of non-placed youth are in need of monitoring and have significant reentry needs for 
services that would be best provided by JR parole [aftercare].” 


“The treatment quality of FFP appears more variable, relative to FFT quality, ranging from low for FFP 
counselors with seemingly limited commitment and fidelity to the FFP approach and moderate for counselors 
who seem to have embraced the FFP model and appear motivated to implement FFP with integrity. 
Hampering treatment quality for FFP is the inadequacy of training, beyond the initial FFP training, the 
supervision structure, and fidelity monitoring procedures” (Waldron, 2020 — see Appendix E for the full FFP 
report). 


Summary of Functional Family Therapy (FFT) Findings 

“As a whole, the FFT therapists appear to be performing at a uniformly exemplary level, given the restricted 
resources available to them. Without exception, all of the FFT therapists participating in the site visits 
associated with this report demonstrated the high levels of knowledge and skills required for effective FFT 
implementation. Treatment dosages for youth and families who receive FFT appear quite good, with number 
of therapy sessions and rates of treatment retention and completion for FFT providing solid indicators of FFT 
quality.” 


“The structure of placing FFT therapists in the role of [clinically] supervising FFP counselors is detrimental to 
both FFT and FFP implementation quality because FFT therapists are responsible for overseeing FFP fidelity for 
counselors who do not directly report to them and who are formally supervised by others who conduct their 
performance evaluations. This situation gives FFP counselors the option to follow or reject the guidance of FFT 
therapists, limiting the impact of therapists’ time and skills. ...FFT quality is also diminished somewhat by the 
lack of systematic training for all therapists in all elements of the ITM” (i.e. DBT and ART), (Waldron, 2020 — 
see Appendix E for the full FFT report). 
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Summary of Aggression Replacement Training (ART) Findings 

JR uses the Integrated Treatment Assessment (ITA) to determine eligibility for ART. “None of the research 
reviewed indicates that the current eligibility criteria is predictive of future violent behavior, which would 
require ART as an appropriate treatment. The current program is using an assessment for eligibility, and the 
assessment is being used appropriately, however, JR has not determined that the eligibility criteria that have 
been selected are appropriate.” 


“JR appears to be implementing the treatment according to the design. There are strong training and quality 
assurance protocols in place. In terms of dosage, all those who start the program receive the same dosage, 
which is three sessions per week for 10 weeks. It is not clear whether youth are receiving the right dosage. It is 
likely that some youth require more treatment and some less, however, the current design of ART does not 
allow for this type of dosage variation... There was some reporting that ART is only allowed 45 minutes for 
sessions in some places due to school schedules. This would result in a lower dosage than intended. There is 
variation by location in terms of when ART is administered, but the standards and quality assurance for the 
program is consistent across location” (Fox, 2020 — see Appendix F for the full ART report). 


Summary of Specialized Sex Offender Treatment (SOT) Findings 

“The DCYF program, particularly as delivered in its inpatient facilities, emphasizes a strong skills-based 
approach using empirically supported treatments such as Dialectical Behavior Therapy and Aggression 
Replacement Training... Case notes reflected treatment plans, which in turn reflected the medical and/or 
psychiatric evaluations from which they were drawn as well as the legal documents for each youth. Case notes 
were generally well written and the treatment plans themselves reflected and enabled an individualized 
treatment approach for each youth in treatment... On balance, there is much to be proud of within this branch 
of DCYF services. 


The number one concern expressed by all interviewed is that the treatment provided within the inpatient 
components [DBT and ART] of the program is not provided by licensed clinicians. Although the programs are 
assisted by psychologists, the treatment itself is delivered by people who do not have specialized training in 
psychotherapy... The second most cited concern is staff turnover”? (Prescott, 2020 — see Appendix G for the 
full SOT report). 


Summary of Substance Use Disorder (SUD) Treatment Findings 

“It does not appear that youth are being appropriately matched with treatment based on need. While there is 
a range of levels of care provided across the institutions, each institution only provides one or two levels of 
care that may be dependent on what substance is being used. 


Assignment of youth to particular institutions is made based on multiple considerations, including age, gender, 
sentence length and other considerations, of which SUD prevalence/severity is not one. Youth in need of SUD 
treatment get the level of treatment offered at the institution to which they were remanded, regardless of 


3 Specialized sex offender treatment for youth in state community facilities and for youth on parole supervision are provided 
through contracted services from community providers. 
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their level of need/severity. In many cases, this means youth get less treatment than they need, in some cases, 
more treatment than they need and in other cases, no treatment at all.” 


‘JR appears to be implementing the chosen treatments as well as possible given the constraints of the 
settings. Treatments are generally manualized and delivered in the context of an individualized treatment 
plan. Staff are passionate about providing high quality SUD care. However, systemic issues interfere with the 
ability of JR to provide optimal treatment. For example, it was noted that staffing was often a problem. As 
noted, at the time of the site visit, Echo Glen Children’s Center (EGCC) was not providing intensive outpatient 
due to staffing challenges. As long as this is the case, no girls in a JR institution receive SUD treatment... There 
was no formal quality assurance plan identified at any of the sites visited” (Stoner, 2020 — see Appendix H for 
the full SUD treatment report). 


Summary of Special Mental Health (MH) Treatment Findings 

“While all of the mental health treatment providers were clearly motivated, compassionate and conscientious, 
they also all reflected on the inadequacy of the system (lack of sufficient staff, high complexity of needs) for 
delivering high quality mental health treatment. Neither approach (short term or through the entire term) 
appeared to be guided by symptom reduction (with the exception of TF-CBT which could be extended if youth 
symptoms were not resolving).” 


“A strength of mental health treatment across all institutions is the effort to coordinate the medical and 
psychosocial treatment of mental health needs through team-based planning and ongoing coordination. 
Treatment plans are reviewed with mental health therapists, coordinators, psychiatrists and medical directors 
in the three institutions.”° 


“Mental health therapists serving youth in the JR institutions are not being routinely trained in best practice 
clinical treatment standards. The fellowship rotation at Echo Glen ensures a number of youth have access to 
treatment by residents who are being exposed to evidence-based treatment strategies through other 
rotations but training for JR therapists is otherwise limited to what the clinician was trained to do prior to hire 
and whatever clinical strategies they are motivated to learn more about as part of their ongoing clinical 
education. The treatment approach is generally eclectic and therapists are being called upon to address a very 
wide range of needs without adequate training and consultation support... There did not appear to be a 
quality assurance plan for monitoring the treatment of specific mental health disorders” (Walker, 2020 — see 
Appendix | for the full Specialized MH Treatment report). 


Summary of Suicide and Self-Harm Prevention (SSP) Findings 


“The policy that governs the procedural practice of assessing, managing and treating suicidal and self-harming 
youth is comprehensive, sophisticated, and contains numerous safeguards to ensure each at-risk youth’s 
situation is carefully assessed. All staff working with youth are required to complete multiple trainings on 
suicide and self-harm prevention... This training is very thorough, comprehensive, and is likely more than most 
licensed clinicians receive in graduate school. Brilliantly, employees are expected to study and memorize risk 


4 Substance use disorder treatment for youth in state community facilities and for youth on parole supervision are provided through 
contracted services from community providers. 
5 Specialized mental health services for youth in state community facilities and for youth on parole supervision are provided through 
contracted services from community providers. 
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and protective factors before arriving at their training... These training approaches are sophisticated and 
rigorous.” 


“The treatment plan to address suicidal and non-suicidal self-injurious behavior requires sophisticated 
behavioral procedures that are contained within Dialectical Behavior Therapy (DBT)... Staff shortages and staff 
turn-over have significantly compromised JR’s ability to train up its workforce in these and other procedures 
and to maintain whatever training gains it makes. Non-competitive wages make it particularly challenging to 
recruit personnel at all levels (AA, BA, MA, PhD). The situation is even more dire in rural areas. Without staff 
retention and comprehensive training, consultation and supervision [related to DBT], it is difficult how 
counselors can be expected to actually imp/ement an effective treatment plan that reduces suicide and self- 
harm risks. Blocking procedures and the [Suicide Precaution Level] SPL procedures will ensure that those at 
risk in the moment do not have access to lethal means and methods to cause self-injury or death by suicide. 
This is different, however, than having a carefully developed treatment plan...that will serve them well after 
they leave JR” (Dimeff, 2020b — see Appendix J for the full SSP report). 


Finding a Path Forward: Recommendations 

The goal is to create an ITM that can work effectively today, and that can also expand as additional resources 
are added. The basic JR ITM approach is sound, however, significant improvements to integration and 
implementation are needed. Many of the recommended changes to the ITM can be made without additional 
funding. The agency must focus on treatment implementation, which includes having the correct eligibility 
criteria, high quality treatment and providing the correct dosage based on the client’s level of need. As 
implementation improves, we expect that this will have a direct impact on improved outcomes. There are 
numerous treatment area-specific recommendations in the assessments located in the full consultant reports, 
which are provided in the Appendix, and a list of some of the top area-specific recommendations can be found 
in the table at the end of this section. For those interested in a specific treatment area, we encourage you to 
read the assessment report on that treatment. Experts wrote these comprehensive and insightful reports. The 
following recommendations are seen as the top priorities to improve JR’s ITM implementation. 


Recommendation 1: Realign Organizational Structure to Create Clear ITM Oversight 

This recommendation is the top priority. The remaining recommendations are not likely to be successful 
without a clear oversight and management structure for the ITM. DCYF should re-align the organizational 
structure to create a clear decision making process and monitoring of the ITM. This includes a more clear plan 
for how DCYF will align QA practices, treatment options and assessment in support of the ITM. The current 
structure, where treatment administrators are spread throughout the agency and there is no clear oversight 
or accountability of assessments (recommendation 2) or treatment quality (recommendation 5), is hindering 
the agency’s ITM implementation. Further, under the current structure there is no clear process in place to 
correct the issues that have been identified in this report. A defined organizational structure that prioritizes 
treatment will provide the structure for an accountable and transparent process to be implemented. DCYF 
should take this opportunity to define what integrated treatment means, and better align the resources 
necessary for an efficient, effective and culturally responsive treatment model. By aligning the treatment 
resources, the agency will be in a better position to create an ITM with feedback loops for continuous 
improvement and monitoring. 


JR has selected the ITM as its strategy to rehabilitate youth. The organizational design should support that 
strategy. The first step in operationalizing this recommendation should be a more formal assessment of how 
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the strategy, structure, processes, rewards and people are all aligned, or misaligned. This can be done through 
a STAR model assessment to determine if the agency is designed to accomplish its goal of effective treatment 
and rehabilitation (Galbraith, 2014). 


The more appropriate organizational structure would view treatment through the lens of the ITM and RNR 
principles. Where youth are being assessed, highest risk youth are prioritized and youth are being matched 
with treatments based on their needs. Treatment access should not depend on the facility the youth is placed 
in. Instead, the youth should be placed in the facility that can best meet the youth’s treatment and other 
needs. By realigning the JR organizational structure, which should include the integration of assessment, QA 
and treatment oversight into the broader DCYF plan, the treatment model can move forward with the tasks of 
integration, monitoring and refinement that are outlined in the consultant reports. An integrated and 
realigned organizational structure would include one person (or a clearly defined and empowered committee) 
who is ultimately accountable for the ITM implementation. They would oversee all the treatment 
administrators, and the implementation and oversight of the risk and needs assessments (recommendation 2). 
Further, the ITM oversight structure would also provide the clinical oversight for those providing case 
management (recommendation 4). 


Recommendation 2: Create an Accountability Structure for Risk and Needs Assessment 
There has never been a full integration of the assessments and the treatment programs in the JR ITM. First, 
there needs to be a process of identifying and prioritizing the top needs of a youth when they enter custody. 
Every youth’s top treatment needs should be identified during the initial intake process, and then referrals 
made for both the treatment type and the dosage needed for each youth. As part of this process, eligibility 
criteria for all the treatment options and dosage levels needs to be clearly established and routinely tested. 
The assessment process, and clearly matching youth to the appropriate treatment, is essential for their future 
SUCCESS. 


The fact that JR continues to use risk assessment tools that have not been validated is a major concern. All risk 
assessments that make decisions about youth care must be routinely validated and tested for reliability. DCYF 
should identify or develop an administrative position or function to oversee assessments. “The new 
administrative position would oversee new and ongoing training for existing (and newly adopted) 
assessments. The position would also be responsible for conducting quality assurance reviews of case plans 
derived from assessment activities insuring that they are driven by the RNR (Risk Need and Responsivity) 
principles. Likewise, the position would oversee regularly scheduled tests of validity for all assessments in use, 
as well as efforts to ensure acceptable levels of interrater agreement and interrater reliability, and would also 
oversee the adoption and implementation of a standardized responsivity assessment process. Under the 
assumption that assessment activities should drive, and be integrated with the treatment models, the Director 
[or Administrator] of Assessment would be ultimately responsible for making sure that true integration occurs. 
Assessment procedures are the cornerstone of any effective intervention model, and as such the integrity of 
the information that is gathered, the way the information is used and the extent to which the information 
undergirds every part of the system is paramount” (Holsinger and Holsinger, 2020 — see Appendix C). 
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No matter how the agency decides to oversee the assessments, all assessments, particularly those used to 
make decisions related to facility placement and length of incarceration, should be routinely validated.® “[The 
RAR, RAI, RACF] continued use should be based in large part on both the results of tests of predictive validity 
as well as the overall agency objectives” (Holsinger and Holsinger, 2020 — see Appendix C). 


Recommendation 3: Provide Transition Support to All Youth Leaving Residential Facilities 

In the recently released DCYF draft strategic plan, one of the agency priorities is to “create successful 
transitions into adulthood for youth and young adults in our care.” This successful transition for JR youth is 
largely dependent on the successful process of reentry from a residential facility back into the community. The 
reentry planning process begins right when youth are admitted to a JR facility. It is paramount that JR support 
the successful transfer and maintenance of gains made by youth while under residential supervision (e.g. 
treatment progress, education improvement and housing stability) into the community setting (including the 
Child Welfare system). 


Parole is an important component of successful community reentry. About half of youth released do not have 
any support from JR when they are released from a residential facility and transition back into the community. 
Even without additional resources, JR should consider how all youth might receive community supports. 
“Although this would require a systemic change at the legislative level, such a change could be initiated at a 
minimal level without an increase in funding... Monitoring and providing services to [all JR] youth would also 
likely result in substantial savings to the state through decreased recidivism and reincarceration. Although 
attempting such a change without an increase in funding would not produce optimal results, the empowering 
of JR parole to allocate resources across the single continuum of need would help guide the application of 
current resources more efficiently” (Waldron, 2020 — see Appendix E). 


“Supervising all youth transitioning to the community would double the overall number of youth served, but 
would not require a doubling of the workload for JR parole staff. A substantial number of lower risk youth 
would need less monitoring and could be supervised with monthly FFP check-ins and brief phone contacts. 
Similarly, youth with moderate risk, including those who would have been placed in JR parole and those who 
would have been released without JR parole placement, could be supervised with bi-weekly FFP check-ins, 
with phone contacts as needed, while higher risk youth could receive services as they are now provided by JR 
parole. Allowing adjustments in length and intensity of service to be made internally by JR parole, based on 
validated assessments, would significantly enhance the efficiency and effectiveness of services across all 
youth. Because all youth not placed in JR parole are released from incarceration with some arrangement for 
housing, funding for housing services would not necessarily need to increase” (Waldron, 2020 — see Appendix 
E). 


JR, with the support of the Legislature, must find innovative ways to ensure all youth can transition smoothly 
into the community. For example, all youth have a residential sentencing range and are required to spend a 
minimum amount of time in a residential facility. The time between the minimum and maximum sentencing 
range could be seen as the transition period. After the minimum date, JR could begin to transition the youth 
back to their home community, with parole services and electronic home monitoring, returning the youth 
back to a facility if they violate certain rules. The important factor here is that all youth need support as they 


5 Through this ITM assessment process, DCYF has engaged the external consultants to begin risk assessment validation work of the 
RAR, RAI, and RACF. 
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return to their community and JR should not release youth from an institution directly back to the community, 
without some level of support for a successful transition. This process must be guided by data and the 
research on what is most effective. 


Recommendation 4: Transition to “DBT for Some,” and Develop Specialized Therapists in 


Residential Facilities 

It is clear that adjustments are needed so that youth receive the services outlined by agency standards, 
particularly related to the main residential treatment model, DBT. Although there are staffing issues such as 
turnover and staffing levels that need to be addressed, there are changes the agency can make now to 
implement DBT in a manner that enables JR professionals to provide higher quality treatment to youth. 
Specifically, JR should transition to providing full DBT only to those who need it the most. Second, JR should 
begin transitioning toward specializing responsibility for providing individual counseling sessions and skills 
groups. Staff are not adequately trained to administer the treatment and turnover is high. It takes a significant 
amount of resources to sufficiently train staff to effectively perform their job duties and many leave before 
they are proficient. 


As outlined by Dimeff (2020a, Appendix D), “JR should reconsider comprehensive DBT for some (vs. 
comprehensive DBT for all). It is a noble effort to seek to provide comprehensive DBT for all youth, particularly 
in light of the real challenges to hiring and retention of staff. While DBT for all may be the ideal, it may also not 
be affordable (and therefore realistic). It is better to provide comprehensive DBT to those who need it the 
most rather than “DBT Lite” for all. Clear criteria could easily be established to determine those who most 
need DBT counseling to fidelity. A small cohort of clinicians/counselors could then be trained and supervised 
to ensure they have the capability of providing DBT to fidelity in the individual counseling mode. All other 
residents could receive DBT Lite (EA, DBT skills). Those selected to provide 1:1 DBT to fidelity may also have 
additional incentives to tackle work with the most difficult of youth (e.g. more training opportunities, 
opportunity for more competitive wages, protected time for consultation team).” 


DBT can be a valuable treatment, however, not all youth need it and certainly not all youth need the same 
dosage. This decision to attempt to give everyone adherent DBT has resulted in very few youth actually 
receiving the treatment with high quality (Fox, Miksicek and Veele, 2019). 


Next, JR should have specialized therapists provide DBT and case management. JR is asking too many staff to 
take on too many tasks. Currently, JR expects staff to be proficient in three of the four DBT treatment modes: 
coaching on the floor, individual sessions and skills groups. The training requirements are significant in order 
to bring a staff up to speed in all three areas. It might take years for a professional to develop proficiency in all 
modes. Consequently, staff are providing DBT modes without sufficient training and quality improvement 
support. They are leading skills groups with insufficient training in DBT skills and group management; coaching 
youth on the floor without enough training in behaviorism and goal-directed coaching; and conducting 
individual sessions without sufficient training in engagement, motivation and behavior change strategies. The 
current implementation strategy is not producing the quantity or quality of treatment necessary for effective 
DBT treatment dosage. This recommendation will help improve treatment for youth in JR and will, hopefully, 
align workers’ expectations with their compensation (Hyzer Group, 2019a). 


Leadership should focus and specialize the scope of responsibilities for the current job classifications within JR, 
and consider adding a classification for specialized staff who provide individual counseling, case management 
and reentry planning. As an example, the staff with specialist classification would be responsible for a larger 
individual counseling caseload, and would have limited or no duties managing the floor. This group of 
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specialists will receive intensive training in DBT case management and reentry support. Having DBT provided 

by a specialist would allow others in the living units to focus solely on coaching youth and managing the living 
unit environment. The specialists would be accountable to the ITM oversight structure (see recommendation 
1); 


The agency could realize cost savings by targeting specific training and quality assurance activities with smaller 
groups of specialists based on their scope of responsibility. Training everyone on everything remains a 
significant logistical and financial challenge. Tailoring and prioritizing trainings for targeted specialists 
(therapists, group facilitators, and coaches) will accelerate competency and mastery, resulting in a multitude 
of benefits to the agency, facilities, employees and clients. This new approach could contribute to increased 
staff morale and retention by empowering and supporting staff to focus their development, with considerably 
more targeted and direct support. Staff specialization will allow staff to feel more capable once they are able 
to master their responsibilities. 


Recommendation Five: Adopt a Uniform and Clear Quality Assurance Program Across the 
ITM 

Currently, there is a lack of consistent monitoring of treatment implementation. In the absence of data, it 
becomes difficult to create a system of accountability and transparency. To that end, JR needs to create a 
uniform quality assurance and continuous quality improvement model for the ITM. Currently, the ITM Quality 
Assurance team is only dedicated to conducting QA for a portion of DBT. In order to monitor the ITM, so that 
agency leadership can see both the quality and quantity of treatment being delivered, the new QA structure 
should include a number of key features. First, a treatment dashboard needs to be developed. Data on both 
treatment need and treatment access could be monitored in near real time. Second, the role of quality 
assurance should be seen as separate from program oversight. Often the administrator for the program or 
program area is also the same person who provides quality assurance monitoring. This creates a situation 
where treatment administrators are being asked to report on the quality of their own work. Third, JR needs a 
more routine and standardized quality assurance process. Currently, each program in the ITM has a different 
QA structure and reporting process. QA needs to be consistent so that changes over time can be identified, 
but it also has to be feasible given the limited resources. JR should explore the Standardized Program 
Evaluation Protocol (SPEP) or a similar standardized tool that will allow for the routine monitoring of programs 
with the same set of protocols. 


There is significant institutional knowledge among JR staff about what works, and what does not work, for 
treatment and rehabilitation of juveniles. The youth in JR have many treatment needs and are at high risk for 
future offending. Because of this, an efficient system of assessment that informs treatment prioritization is 
essential. Leadership (both within the agency and the Legislature) must ensure that our most valuable asset, 
the staff, are supported by a system that is designed for success. This will result in lower recidivism, fewer 
victims and youth leading lives that are more successful in the long term. The adoption of the 
recommendations listed in this report will move JR into a new phase of treatment implementation that is 
more effective, efficient and data-driven. 
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Top Area-Specific Recommendations From Consultant Reports 


Recommendation 


Number* 


Focus Area 


Recommendation 


Develop a point person (e.g. a Director [Administrator] of 
Assessment) or function and a small administrative 


Therapy (FFT) 


C.1 Assessment system structure that would be dedicated solely to implementing, 
monitoring and driving everything related to the system’s 
assessment procedures. 

The continued use of the RAR, RAI and RACF should be 
C.2 Assessment system based in large part on the results of their tests of predictive 
validity. 
If the RAR, RAI and RACF do possess current predictive 
C3 Mec ceementeyemn validity, consider utilizing them as instruments that dictate 
, intensity of supervision and/or the intensity of the 
residential placement. 
Give priority to the ITA in order to drive more aspects of 
the treatment model currently in place. Ensure that the ITA 

C.4 Assessment system and the information that it renders is fully understood and 
utilized by any and all staff that work with the youth in any 
Capacity. 

Bialecrsatbohevicr Reduce staff shortage and improve retention. stall 

D.1 Therapy (DBT) shortages make it difficult for staff to attend trainings and 
have the necessary time to do what is required well. 

D2 Dialectical Behavior Reconsider comprehensive DBT for some (instead of 

° Therapy (DBT) attempting DBT for all) 
Dialectical Behavior Offer comprehensive, intensive training in DBT over the 

D.3 

Therapy (DBT) course of a year. 
; . Parole could have a far greater impact if each region 
Functional Family Parole a : 7 

E.1 (FFP) maintained oversight and supervision of a// youth re- 
entering their communities. 

£2 Functional Family Parole A more rigorous fidelity monitoring measure is needed to 

: (FFP) improve the quality of FFP. 
. . Additional FFP counselors are needed at all sites to 
Functional Family Parole | . ; ; : 
E.3 (FFP) improve the frequency and duration of FFP sessions with 
youth and families. 
E.4 neon Eun Funding is needed to increase FFT and FFP staff salaries. 
Therapy (FFT) 

E.5 punevonar ly Provide additional resources to FFT therapists for travel. 
Therapy (FFT) 
Functional Family Parole Develop checklists, procedures and/or benchmarks to 

E.6 (FFP)/Functional Family monitor cross-site consistency of FFP and FFT 


implementation. 
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F ional Family Parol 
ae ee ee J abe : Allow all FFT therapists and FFP counselors to attend or 
a PERE Onedovab arity “audit” ongoing trainings for DBT and ART 
Therapy (FFT) BOIns 6 ; 
Functional Family Parole | Add capacity to conduct data analyses to monitor program 
E.8 (FFP)/Functional Family | performance (treatment quality, quality assurance, FFT and 
Therapy (FFT) FFP outcomes) internally. 
: Refine the ART eligibility criteria so that it is based on 
Aggression . 
F4 eee mere oer evidence accumulated over the past 10 years of 
: P 6 implementation, and then focus ART on those with the 
(ART) : i 
greatest need and the highest risk. 
Aggression Reconsider how ART is being implemented to ensure a full 
F.2 Replacement Training hour for programming for ART sessions, to ensure 
(ART) implementation fidelity. 
Implement a process that allows for closely monitored 
Aggression variations of ART programming. For example, JR could test 
F.3 Replacement Training a shortened version of ART for youth who have the 
(ART) assessed need and are high risk, but do not have a very 
long sentence. 
SBETESSION ae Set up a curriculum review committee for ART, to make 
F.4 Replacement Training ' : 
sure lessons can continually improve. 
(ART) 
Specialized Sex Advancements in the areas of trauma-informed care, 
G.1 Offender Treatment motivational enhancement and the further development of 
(SOT) approach goals would be welcome. 
eheciallzed3ex On-site clinicians for treatment provision in the residential 
G.2 Offender Treatment fee e iG Be dee| 
(SOT) oe 
HA Substance Use Disorder | Provide SUD treatment staff with a new title (and higher 
° Treatment (SUD) pay) that reflects their different role within the institutions. 
Continuity of care could be enhanced by providing SUD 
: treatment records to community facilities when youth 
Substance Use Disorder ; ieee : 
H.2 arrive there from institutions and to community treatment 
Treatment (SUD) : : ; : 
providers with whom paroled youth continue their 
treatment. 
: Make regular use of a SUD treatment oversight committee 
Substance Use Disorder ; . : 
H.3 to guide the implementation of SUD assessment and 
Treatment (SUD) ee ee es 
treatment in juvenile justice settings in the state. 
H.4 Substance Use Disorder | Each institution should provide multiple levels of care to 
‘ Treatment (SUD) better match youth to SUD treatment according to need. 
: : icul id also i 
Suhesnce Ue OeerieE Funding for altemative curricula could also improve: 
H.5 Treatment (SUD) treatment quality if other treatments can be identified that 
better fit the constraints of correctional settings. 
F Refresher training for providers in their treatment models 
Substance Use Disorder eee : : : 
H.6 could enhance fidelity. Curriculum review by credentialed 
Treatment (SUD) : ae 
treatment trainers could also be beneficial. 
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Restructure the delivery of mental health treatment to 
: h ast ith 
Specialized Mental provide more stepped care approac es beginning wit 
1.1 group-based treatment facilitated by mental health staff 
Health Treatment (MH) . 
and then moving to one on one treatment for youth who 
continue to display consistent and concerning behaviors. 
Seadelod ianel Improve clinical Sel NR) by either a) involving regional 
1.2 mental health coordinators from the youth’s home 
Health Treatment (MH) ; . ae ae 
community; or b) improve telemedicine facilities. 
Ensure mental health treatment plans and targets of 
13 Specialized Mental therapy are integrated in milieu behavioral goals and 
, Health Treatment (MH) | planning by adapting current family system models to work 
with line staff in cottages and units. 
Reduce Suicide & Self Screen (SSS) interview process. It is 
M4 Suicide and Self-harm recommended that JR consider convening a task force 
‘ Prevention (SSP) comprised of all relevant stakeholders (including youth) 
and outside suicide experts to streamline the method. 
Ensure adequacy of training for DBT core competencies. 
Th | | ill | Il h 
Suinideand SeleHarin e only way a counse OF will be ab eto actua y treat that 
J.2 . which they assessed using the SSS is by receiving 
Prevention (SSP) . ies oe 
comprehensive training in the treatment procedures, 
including ongoing consultation and supervision. 
etiCendise chara ere consider providing Designated Suicide Prevention 
J.3 : Specialist (DSPS) option to deviate from standard 
Prevention (SSP) 
procedure. 
SuididerandSele nari Consider having a designated suicide expert available for 
J.4 : more complex cases and to conduct quality assurance 
Prevention (SSP) . 
review of SSS. 


* The letter before the recommendation number indicates the appendix where the full report can be 
found. Please see the full consultant reports for more details on each of these recommendations. 
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Appendix A: List of questions related to ITM assessment system 


Questions for Assessment System: 


I 


Considering the JR assessment system, are the assessments appropriate and integrated 
with the treatment model? 
Are youth being appropriately matched with treatment based on assessed risk and 
need? 

a. What assessments are being used for program eligibility? 

b. Are the assessments being used appropriately? 
Does the current quality assurance plan adequately monitor assessment performance? 
Please provide a statement of qualifications. 
What recommendations do you have to improve the assessment system in JR, given the 
current level of resources? 
What additional resources would you recommend to improve the assessment system 
and integration? 
JR will be serving youth up to age 25 in the future. What advice related to the 
assessment system would you give the agency as they prepare to provide this treatment 
for older youth? 
Please provide a statement of qualifications. 


Appendix B: List of questions related to ITM treatment areas 


Questions for treatment categories: 


1. Are youth being appropriately matched with treatment based on need? 
a. What assessment is being used for program eligibility? 
b. Is the assessment being used appropriately? 

2. Is the treatment high quality? 

How well is JR implementing this treatment? 

Are youth receiving the appropriate dosage? 

Do staff receive the correct type and amount of training? 

Does there seem to be variation by location? (between institutions, between 

institutions and community facilities) 

3. How well is the treatment integrated with the assessments, other treatments, and 
reentry planning? 

4. Does the current quality assurance plan adequately measure the treatment model? 

5. Do you see any evidence that would concern you that treatment access varies by race, 
ethnicity, age, or gender? 

6. What recommendations do you have to improve implementation given the current level 
of resources? 

7. What additional resources would you recommend to improve treatment quality and 
integration? 

8. JR will be serving youth up to age 25 in the future. What advice would you give the 
agency as they prepare to provide this treatment for older youth? 

9. Please provide a statement of qualifications. 


Qooo 


Appendix C: Fidelity Assessment of ITM Assessment System 
by Dr. Alex Holsinger and Dr. Kristi Holsinger 


Fidelity Assessment of ITM/JR Assessment System 


Alexander Holsinger, PhD 
Kristi Holsinger, PhD 
April 2020 


Executive Summary 


Considering the JR assessment system are the assessments appropriate and integrated with 
the treatment model? 

The RAR, RAI, RACF and ITA served as the primary focus for this question. Each of the 
instruments were reviewed for face validity, item weighting, and scoring procedures with 
potential liabilities noted. Of the four instruments the ITA holds the greatest potential to drive 
case planning, youth classification (on risk and need), treatment models, and the assessment of 
change over time. The ITA is a lengthy and comprehensive assessment that measures a large 
number of relevant criminogenic domains in depth, offering both risk/need and protective 
factors. In addition, the ITA is conducted via a one-on-one interview as well as the review of 
official records and databases. The other three instruments are based largely on delinquent 
history, and do include some items from other relevant criminogenic domains such as 
substance use/abuse, but not to a degree that could drive an individual treatment model. While 
the three assessments may possess predictive validity (if their use is to continue new validation 
studies are recommended), their use is likely better served outside of a treatment environment 
(particularly in light of the potential for the ITA). 


While the ITA does provide a great deal of important information, it does not appear that it is 
currently integrated into the treatment model as much as it could be. For example, treatment 
activities are driven largely by (pod/house) placement, which appears to be driven by 
procedures that do not involve the ITA to a substantial degree. Overall the potential for the ITA 
is not currently being realized. 


Are youth being appropriately matched with treatment based on assessed risk and need? 

As noted above, the ITA is not currently driving the treatment model to the extent that it could. 
Best practices in correctional interventions state that delinquent youth should be referred to 
effective criminogenic need-specific programming (e.g., cognitive behavioral interventions), and 
that the referrals should be based on a comprehensive dynamic risk/need assessment. In 
addition, youth should only receive programming that applies to their unique set of 
criminogenic needs. Further, responsivity factors should be assessed and utilized as well in 
order to match youth to programming and therapists that the fit the youth best regarding their 
profile (e.g., learning style, motivation, readiness for change and other responsivity factors). 
Currently youth are being assessed by a psychologist who makes residential placements based 
on the information gathered. Residential placement in turn drives the treatment activities that 
occur. As a result, the link between criminogenic risk/need activities (e.g., via the ITA) and 
programmatic matching is not as strong as it could be. 


Does the current quality assurance plan adequately monitor assessment performance? 
Quality assurance activities are in place, but there is some room for improvement (though the 
improvement may be due to the need for additional resources — see below). Staff that conduct 
the ITA assessments go through a rigorous training led by a seasoned master trainer with an 
immense amount of experience using the ITA, as well as experience in relevant training 
activities. The initial training does include inter-rater agreement tests as well, that have 
revealed a high rate of agreement. The quality assurance activities would benefit from 
additional staff that could assist with one-on-one observation of staff that conduct the ITA ona 
regular schedule (e.g., once per year), “booster-session” trainings on a tri-yearly basis, and 
additional inter-rater agreement testing. In addition, the ITA should be subjected to a regularly 
occurring (e.g., every 3 to 5 years depending on circumstances) test of predictive validity using 
relevant and varied outcomes. 


What recommendations do you have to improve the assessment system in JR, given the 
current level of resources? 

e If the RAR, RAI, and RACF will continue to be used, consider limiting their use to other 
parts of the justice system. Their continued use should be based in large part on the 
results of their tests of predictive validity. 

e If the RAR, RAI, and RACF do possess current predictive validity consider utilizing them 
as instruments that dictate intensity of supervision and/or the intensity of the 
residential placement. 

e Give priority to the ITA in order to drive more aspects of the treatment model currently 
in place. Insure that the ITA and the information that it renders is fully understood by, 
and utilized by, any and all staff that work with the youth in any capacity. 


What additional resources would you recommend to improve the assessment system and 
integration? 

Consider developing a point person (e.g., a Director of Assessment) and a small administrative 
structure that would be dedicated solely to implementing, monitoring, and driving everything 
related to the system’s assessment procedures. The new administrative position would oversee 
new and ongoing training for existing (and newly adopted) assessments. The position would 
also conduct quality assurance reviews of case plans derived from assessment activities insuring 
that they are driven by the RNR (Risk Need and Responsivity) principles. Likewise, the position 
would oversee regularly scheduled tests of validity for all assessments in use, and would also 
oversee the adoption and implementation of a standardized responsivity assessment process. 


Additional resources would also benefit the expansion of treatment and rehabilitative activities 
within the JR system. Additional treatment programming will likely be necessary in order to 
adequately address criminogenic needs, particularly if the ITA were used to more fully drive 
case planning and programmatic placement. In short, it is likely that a fuller utilization of the 
ITA will reveal the need for more treatment capacity. Additional resources for treatment 
programming should also include allowances for processes that will regularly and quantitatively 
assess the quality (i.e., integrity or fidelity) of the programming. 


JR will be serving youth up to age 25 in the future. What advice related to the assessment 
system would you give the agency as they prepare to provide this treatment for older youth? 
It will very likely be necessary to adopt, implement, and test the validity of a tool designed to 
assess the adult offender population. Tools such as the LSI-R, the COMPAS, the ORAS, or 
another comprehensive tool of sufficient breadth and depth that has a high degree of 
demonstrated predictive validity should be considered. Once adopted best practices in offender 
assessment as they relate to training, implementation, and tests of validity on the local 
population should be observed. 


It is possible that portions of the ITA could be used on the young adult offender population, but 
it would be necessary to go to great lengths to ensure that the tool (and/or the portions 
therein) do indeed adequately predict the desired outcomes for young adult offenders aged 18 
to 25. 


Integrating assessment activities and information into treatment models: Washington State JR 


Submitted by Alexander M. Holsinger, Ph.D. & Kristi Holsinger, Ph.D. 


In an effort to improve the treatment the Washington State Department of Children, Youth & 
Families (DCYF) provides to youth in Juvenile Rehabilitation (JR), the Washington State Legislature 
designated funding for a fidelity assessment of their Integrated Treatment Model (ITM). A significant 
component of the ITM is the Juvenile Rehabilitation Assessment System which relies on a variety of 
assessment instruments and practices that guide decision making for youth. 

This report is the result of an extensive review of written protocols, published reports, 
descriptive data, and a site visit conducted in early January, 2020. Specifically, the researchers visited 
the Echo Glen Children’s Center, the Green Hill School and JR headquarters, conducting information 
gathering sessions with a wide variety of stakeholders, directors, and staff. In this collaborative research, 
we provide our expert opinion and recommendations while addressing specific guiding questions, to 
assist in the work of moving to more efficient and accurate approaches in assessment. 

Question 1: Considering the JR assessment system, are the assessments appropriate and integrated 
with the treatment model? 

An effective evidence-based assessment model that drives an effective correctional 
rehabilitative environment should embody three principles: Risk, Need, and Responsivity, or RNR. In 
short, the assessment activities as a whole should offer a valid and objective measure of quantitative 
risk (meaning how “at risk” a youth is, which refers to the likelihood they will engage in delinquent 
behavior in the future). The results of assessment procedures should also present professionals with 
each youth’s unique criminogenic (crime-producing) needs. For example the correctional research 
literature has identified the comprehensive array of criminogenic needs that are directly related to a 
youth’s willingness or compulsion to engage in delinquent behavior. As such assessment activities 
should present a profile unique to each youth that indicates which criminogenic needs are active factors 
within the youth and their environment (both physical, cognitive, and social). Assessment activities 
should also result in knowledge about what strengths and barriers may be active within the youth 
and/or their environment that may have a direct impact on their willingness or ability to engage in 
treatment activities. For example, motivation, treatment readiness, intelligence, and learning disabilities 
are all responsivity factors (though not an exhaustive list — see below for an expanded discussion 


regarding the principle of responsivity). 


In addition to informing the principles or Risk, Need, and Responsivity, effective assessment 
models should involve the standardized and objective collection of dynamic items that in turn inform 
specific assessments and scales or scores that result. Dynamic items are those that are changeable (as 
opposed to static items, which do not change). For example, “how old were you when you first tried 
marijuana?” would be an example of drug use measured in a static way while “over the last six months 
what role has marijuana played in your life?” is an example of assessing the same domain in a dynamic 
fashion. The use of dynamic items and information is very important since they allow for the 
measurement of change over time, and via various intervention methods. Effective assessment models 
will facilitate the measurement of change over time, and offer treatment professionals an idea of 
whether a youth is making progress in treatment. Effective assessment activities should drive 
comprehensive and useful case planning, and should likewise drive placement in housing units, 
placement in various treatment programs, and should allow for appropriate and effective release 
planning as well. It is often necessary for youth to be assessed multiple times particularly when they are 
involved in treatment programming and/or living in residential facilities for extended periods of time 
(e.g., a year or longer). 

Overall there appears to be room for improvement regarding the extent to which assessment 
activities in the State of Washington’s Juvenile Rehabilitation system are integrated into the broader 
treatment model. This report focuses primarily on the Risk Assessment — Recidivism (hereafter RAR), the 
Risk Assessment — Institution (hereafter RAI), Risk assessment — Community Facility (RACF), and the 
Integrated Treatment Assessment (ITA). The first three of the aforementioned assessments (RAR, RAI, 
RACF) were each designed for, and to some extent are used for, specific purposes. For example the RAR? 
utilizes several factors (several static, some dynamic) in order to assess the likelihood of post-release 
recidivism. The RAI was designed to determine the likelihood of a youth being placed within an intensive 
management unit once they are already within an institution. Like the RAR, the RAI is comprised of 
several items, some static and some dynamic. The RACF was designed to determine the likelihood a 
youth will return to an institution once they have been placed in a community-based facility. 

The RAR 

As noted above the RAR consists of some static and dynamic items, most if not all of which have 

at least some face validity regarding the statistical prediction of recidivism. The items largely assess 


factors that are of some use in a criminal justice/correctional setting. For example knowing whether or 


1 The exact composition of the RAR differs for male and female juveniles, and as such special attention should be 
paid toward insuring gender bias in decision making does not result. 
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not an individual has a history of assault or facility non-compliance can be useful in making placement 
(e.g., housing, roommate) and other decisions. As a whole the information contained within the RAR is 
not broad nor deep enough in order to inform case planning or drive a treatment model. For example, 
substance use is an important domain to measure when delivering services to system-involved 
populations, and the RAR presents just one item scored as “none,” “impairment,” or “unknown.” While 
this item is informed/scored from other parts of the system, on its own this item is of little value in 
terms of an integrated treatment model. 

Overall, the RAR might have some utility as a stand-alone assessment used to assess the 
likelihood of recidivism in some settings (e.g., after release to the community) depending on systemic 
objectives, but the extent to which a treatment model could be informed by the RAR is likely 
substantially limited. In addition, in the event the RAR continues to be used, attention should be paid to 
the way items are measured, and the weight they are given when contributing to the overall score. For 
example, it is very common for jurisdictions and legislatures to struggle greatly with the measurement of 
gang membership. Gang membership has a long history within the criminal justice literature and much 
evidence has shown it to be predictive of delinquent and criminal behavior. At the same time gang 
membership is something that can change over time as does what constitutes membership itself (vs. 
affiliation or association). 

Another issue that may limit the RAR’s utility is the fact that different versions exist for male and 
female youth. There is support in the literature showing that criminogenic predictors may differ 
somewhat for male and female youth, perhaps in type, form, and importance (meaning weight). 
However some challenges to objectivity and fairness may emerge when different criteria and weighting 
are used for the same instrument that informs the same decisions. For example mental health needs are 
factored into the RAR female but not the RAR male. There is evidence that mental health needs are 
indeed prevalent in the male offender population (yet go unmeasured), but moreover, currently there is 
controversy regarding the inclusion of mental/behavioral health issues at all when it comes to the 
prediction of risk, particularly when decisions that affect how long someone might be confined or how 
closely they might be supervised are at stake. Mental and behavioral health are currently regarded more 
as a responsivity consideration as opposed to a risk factor or a criminogenic need. Likewise, the risk of 
suicide is included on the RAR female but not the RAR male. Further, the “SAVY (Aggressive or Not 
Vulnerable)” item is calibrated in a way that makes it possible to overwhelm the scoring scale entirely. 


The “SAVY” can contribute up to 80 points, while all the other items together max out at a total of 150 


points. Having an item weighted so heavily within the context of the rest of the scored items runs the 


risk of negating the purpose of the risk assessment itself, entirely. 


Recommendations: 

e Reconsider the use of the RAR, particularly in light of other assessment processes that are in 
place. 

e = If it is determined that the RAR does have continuing potential utility, consider testing the 
validity of the tool overall using appropriate outcome(s) and a recent dataset. 

e Statistical analyses should consider the utility of different versions based on sex and/or 
ethnicity. 

e If possible statistical analyses should consider the use of other standardized items that display 
better predictive validity than one or more current items. Along similar lines consider replacing 
mental/behavioral health with another item, or eliminate it altogether particularly if the tool 
maintains predictive validity and differentiates levels of risk adequately (and likewise classifies 
youth into workable proportions) without it. 

e The appropriateness of the weighting and scoring of items should be examined. 

The RAI 

As noted above, the RAI was developed in order to offer predictability regarding the likelihood a 
youth will be placed within an intensive management unit, once they have been placed in a facility to 
begin with. Importantly, the RAI should have utility when making placement and other decisions that 
relate to the level of oversight and monitoring of the youth. Ideally the RAI could be used as a risk 
assessment that supports staff decision making that reduces the likelihood isolation will be necessary. 
Intra-institutional placement is an important decision and ideally one that is based on statistically valid 
and objective criteria. It does not appear that the RAI could inform case planning or drive an integrated 
treatment model given a lack of breadth and depth regarding the domains that are captured via the 
instrument. 

Like the RAR, the RAI contains several items that have at least face validity. For example, prior 
assaultive behavior, impulsivity, substance use, criminal history, prior within-facility behavior all have 
both face validity and support in the literature as predictors of institutional behavior. Most of the items 
are static (unchangeable) in nature, while some have dynamic properties. The items have been weighted 


in a manner so each contributes points relative to the conditions underlying the items. Based on prior 


research, the RAI has demonstrated predictive validity and a statistical relationship with relevant 
outcomes (in this case, placement in an intensive management unit within the facility). 

Like the RAR, the RAI might have some specific utility (in the case of the latter, the making of 
intra- or even inter-facility placement and housing decisions). Given the effort that goes into other 
assessment activities (e.g., the ITA) it might be possible to utilize other information that is being 
gathered prior to the scoring of the RAI in order to facilitate earlier decisions based on richer and more 
dynamic information. If it is determined that the RAI should remain in use, the following 
recommendations should be considered: 

e Reconsider the use of the RAI, in exchange for portions of the ITA. 

e = If it is determined the RAI does have remaining utility, reexamine how each item is scored and 
calculated to revisit and assess the tool’s objectivity. 

e = If it is determined the RAI does have remaining utility, conduct a current test of the tool as is, 
using appropriate outcomes. When possible analyses should also include tests using subgroups 
of the youth population (e.g., categories of ethnicity, sex). 

e If possible, statistical analyses should consider the use of other standardized items that may 
display better predictive validity than one or more of the current items on the assessment. 

e The appropriateness of the weighting and scoring of items should be examined. 

The RACF 

The development of the RACF had the specific objective of creating an instrument that could 
reliably predict the statistical likelihood a youth who had been released to a residential facility would 
return to a JR facility. Like the RAR and the RAI, many of the items on the RACF have at least face 
validity, and likewise the RACF contains a mix of dynamic and static factors. Not surprisingly the RACF 
takes into consideration within-community facility behavior (e.g., progress in the facility, problem 
solving skills, type of responsiveness, compliance with rules) and likewise utilizes criminal history items 
such as history of escape, prior commitments, offender type (sex offender), age at admission, and 
substance use/abuse. The RACF does utilize mental/behavioral health needs, and moreover utilizes prior 
mental health placements in the scoring strategy (two different items tapping the same domain). The 
same issues for the RACF emerge as those that presented themselves with the mental health item on 
the RAR female. Mental health needs are largely regarded as a responsivity issue as opposed to a risk 
factor or criminogenic need, making their inclusion in a risk assessment potentially problematic. In 
addition, the extent to which female youth are at least somewhat more likely to have a mental health 


diagnosis may also bias scoring in some instances (while system-involved males often go un- or under- 
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diagnosed). Of particular concern however, given the weight of the item (20 points — making it one of 
two of the most heavily weighted items on the instrument), administrative procedures make it possible 
for youth to be placed in a mental health unit due to space shortages as opposed to an active behavioral 
health need. As such their risk level may be artificially inflated which in turn has implications for the 
youth’s experience and treatment in the community facility. 

As a whole the RACF does have some face validity regarding some of the items that are 
captured, in particular those that are dynamic and relate directly to youths’ behavior. These items may 
have some value as they relate to the measurement of in-program behavior and progress, but only if the 
items are utilized several times over the course of a youth’s stay in the facility and were reconfigured for 
a different use. As it currently stands the Client Behavioral Assessment (hereafter CBA) may already fill 
the need for the measurement of in-program behavior and progress. The following recommendations 
are put forth should the RACF be considered for use going forward: 

e Reconsider whether the CBA (with needed reconfiguration and/or revalidation work using the 
CBA) could fulfill the needs that are currently met by the RACF. 

e If the RACF is retained, conduct a current test of its validity, including an item-by-item analysis, 
using appropriate outcomes. When possible analyses should also include tests using subgroups 
of the youth population (e.g., categories of ethnicity, sex). 

e = If data are available consider the inclusion of other standardized items that may display better 
predictive validity in exchange for items that may warrant replacement. 

e The appropriate weighting and scoring of items should also be examined. 

The ITA 

The ITA is an in-depth and broad assessment protocol designed for use with system-involved 
youth. Approximately 10 criminogenic domains are assessed using over 150 unique items/questions that 
are populated through the use of a lengthy interview process with correctional professionals along with 
checks of official records. Each of the domains are assessed using both static (e.g., unchanging) and 
dynamic (changeable) metrics. The use of dynamic methods makes possible the measurement of change 
over time and via effective programming. 

In addition to identifying the existence and magnitude of specific criminogenic needs, the ITA 
also offers a snapshot of what is termed “risk” and “protective” factors and levels. “Risk” in essence 
means that a particular domain exists in a manner that is conducive to the commission of delinquent 
behavior (in this sense, what is being identified is an area in need of intervention via effective treatment 


programming). “Protective” means that a particular domain has characteristics that are insulating a 


youth from committing delinquent behavior. According to the ITA a particular domain can have both 
“risk” and “protective” factors, which makes possible the creation of comprehensive case planning that 
can drive treatment interventions. Take for example the domain of leisure or free time. A youth might 
have an excess of unstructured, unsupervised free time, which would be a “risky” situation that exposes 
the youth to opportunities conducive to delinquent activity. The youth could also have very specific 
prosocial interests, such as a specific sport, or an activity like model building, that could be capitalized 
on in order to occupy (previously) unstructured free time. The ITA rates all assessed domains on both 
“risk” factors and “protective” factors and does so using the classification labels “high,” “medium” or 
“low” rating, giving the end-user a concise summary of active criminogenic needs, as well as protective 
factors that could be capitalized on at some point. 

When used appropriately, the ITA is an investment via time and expertise. It takes time and 
great attention to detail in order to conduct the necessary interview, verify information via official 
records, score all the items, and complete the summary assessment. The result is potentially worth the 
effort in light of the comprehensive nature of the information gathered, the depth of detail, and the 
form (dynamic) in which the information is gathered and used. As noted above, the ITA can be used for 
case planning, which ideally will detail the specific type of programming that is necessary in order to 
address active criminogenic needs. In addition, within the context of a valid case plan the ITA will reveal 
specific targets that should be met that will address the need, and depending on the severity of the 
need, the ITA should offer an idea as to what the appropriate length of treatment may be (although this 
lattermost point will require additional research and analysis in order to determine specific dosage 
recommendations). 

In addition to an initial case plan the ITA also offers the potential for measuring change over 
time, which can inform both an individual youth’s progress via programming, but can also (with the right 
analyses) offer how effective a program might be in meeting a specific criminogenic need. Because of 
the dynamic nature of many of the items and domains, subsequent assessments can be compared to 
initial baseline scores in order to measure progress or to indicate the need for more intensive 
intervention of a longer duration. Currently the ITA is conducted toward the beginning of a youth’s JR 
stay and again toward the end, however, there may be value in conducting the ITA (or portions of it) 
more often, particularly for youth who are housed with JR for a lengthy period of time, who are likewise 
involved in specific rehabilitative interventions. 

The JR system would benefit from the ITA having more universal recognition among all staff that 


work with youth in any capacity. One of the advantages of any comprehensive assessment is the 


potential that it forms that basis for a common language with which to refer to the corrective and 


rehabilitative activities that youth participate in, in addition to language related to the criminogenic 


sources of antisocial behavior. While the ICI represents a shortened summary of the ITA, it would be 


beneficial for more staff — even those who are not “end users” of the actual assessment itself — to have 


more knowledge of the ITA, how it is done, the methods by which the information that goes into the 


assessment is rendered and of course the information itself. Increasing the “profile” of the ITA 


throughout the JR system will allow for better implementation of the Risk, Need, and to some extent the 


Responsivity principles of effective correctional intervention. To that end, fuller utilization of the ITA’s 


potential will allow for the aforementioned case planning and the identification of behavioral targets, 


and likewise it can be used to inform risk/need classification systems (e.g., high, medium, low) and in 


turn inform housing decisions, treatment referrals, time served decisions, and readiness to engage in or 


emerge from treatment opportunities. In light of the comprehensive nature of the ITA, the resource 


investment, and the assessment’s potential the following recommendations are put forth: 


[e) 


The ITA should serve as the flagship assessment for the JR, and should form the basis of a 


number of functions and likewise inform a number of decisions including but not limited to: 


The building of a comprehensive case plan that identifies the most pressing 
criminogenic needs (targets for treatment), and that also utilizes protective factors. The 
case plan should serve as a living document that guides a youth’s trajectory through JR 
including treatment referrals, mechanisms where by targets for change will be met, and 
benchmarks that reveal progress. 

The risk/need classification system should be used to make housing decisions, insuring 
that youth with similar levels of risk/need and similar criminogenic targets are housed 
together as much as is possible. The classification and housing decisions may also 
include staffing levels where possible (e.g., a higher risk group would have more staff 
present, a lower risk group may possibly have fewer staff). 

The ITA should be used to measure change whenever possible. Change over time may 
require additional administrations of the ITA (either in full or specific portions). Evidence 
of change via rehabilitative intervention should be used to make changes to the case 


plan as well as placement and release decisions. 


The ITA and the concepts that serve as its foundation should be used to inform all staff that deal 
with youth to any degree. As noted above, a good comprehensive risk/need assessment can 


offer a common vocabulary and a common purpose and foundation to a rehabilitative system. 


e Plans for a test of the ITA’s predictive validity should be made and implemented. The ITA may 
have utility in predicting the statistical likelihood of any number of outcomes, including but not 
limited to within-facility infraction, return to a facility (after placement in a community facility), 
and post-release recidivism. 

e The data that are rendered from the ITA should be used to drive needed treatment 
programming. For example, it is possible that the JR does not currently have the capacity to 
address one or more criminogenic needs of the youth in light of prevalence within the 
population. 

Questions 2: Are youth being appropriately matched with treatment based on assessed risk and need? 

The ITA is not currently driving the treatment model to the extent that it could. Best practices in 
correctional intervention specify that a comprehensive risk/need assessment should drive any 
rehabilitative treatment model whereby youth are matched to the appropriate treatment based on the 
results of the assessment. Too often (nationally) youth are offered treatment activities that appear 
appropriate in general, but the treatment is not specific to the needs of the youth. In short, systems 
often offer clients the services that are available regardless of the need profiles of the individual. Ideally 
an assessment like the ITA is used to identify the unique risk/need (including the active domains) for 
each youth, and the youth is in turn referred only to services that are specific to the needs that are 
revealed. As it currently stands, youths’ treatment activities are driven by their housing placement, 
which in turn is driven by psychological assessment. When validated assessment activities and 
information are utilized to make treatment-related placement decisions, youth are more likely to 
receive only the services they need saving costs in the long term by avoiding wasted services. Related, 
risk/need assessment information may reveal the need to increase treatment capacity specific to one or 
more criminogenic needs. “One-size-fits-all” treatment portfolios not only run the risk of wasting scarce 
resources on youth who are not likely to benefit from them, but there is evidence that over- 
programming youth (i.e., delivering unneeded services) may actually make youth more likely to 
recidivate via a violation of the risk and/or need principles. 

Another aspect of matching involves the rehabilitative principle of responsivity. The principle of 
responsivity requires that correctional systems take into account characteristics of the individual that 
may serve as barriers to successful placement (such as a housing placement) or barriers to successful 
engagement in treatment activities. Responsivity characteristics can come in a variety of different forms, 
and are by definition specific to the individual youth, however many of them can be placed into broad 


categories that agencies can consider in the aggregate. By taking responsivity considerations into 


account an agency can better insure that youth placed in specific treatment activities will benefit as 
anticipated. In addition to placement decisions (housing, treatment) responsivity factors can also be 
used to match youth with appropriate groups, as well as youth with individual staff such as correctional 
counselors. 

Some of the most useful responsivity considerations include personality type (e.g., Myers Briggs 
typologies), 1Q, motivation to change or readiness to engage in treatment, reading comprehension, 
maturity, as well as demographic factors. As a basic example, consider treatment curricula such as 
cognitive behavioral treatment materials, which are often written at a specific reading level. If a 
treatment program or agency does not assess reading comprehension, there is a risk that some of the 
youth placed in a group using the curricula will not benefit in the intended manner due to an inability to 
comprehend. A more complex example might involve motivation to change or readiness to engage in 
treatment. Youth often present varying levels of readiness to change their behavior. A lack of motivation 
to engage may come from a variety of sources, but is often due to a failure to recognize that the lifestyle 
is leading to delinquent behavior, attention from law enforcement and corrections, and in turn a loss of 
liberty and pro-social opportunity. When a treatment program or agency intentionally and formally 
assesses motivation to change behavior, it is possible to make better placement decisions, and likewise 
streamline scarce resources. For example, if a group of youth are highly motivated to engage in 
treatment activities and work toward changing their behavior, they are good candidates to engage in 
need-specific programming right away. If on the other hand a group of youth are assessed as not being 
motivated at all (and are often quite the opposite, and are prone to creating disruption if placed in 
treatment activities prematurely) it might be more appropriate to work with them on their motivation 
first, before engaging them in specific treatment activities that are focused on a specific criminogenic 
need. 

In order to make decisions based on responsivity factors, an agency must assess one or more 
responsivity domains. The assessment of responsivity is typically domain-specific and is done via a 
specific assessment. For example the aforementioned personality typing can be done via Myers Briggs 
testing or motivation can be assessed via the Socrates stages of change process. It is generally 
recommended that agency stakeholders determine what responsivity factors are most relevant to the 
population being served and choose a method of assessment(s) accordingly. In the meantime the 
agency may benefit from merely increasing awareness of responsivity and how it impacts treatment 
effectiveness. It is very likely that responsivity concerns are revealed via the ITA interview process, and 


likewise during various other assessment procedures including one-on-one time with agency 
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psychologists. As such, awareness of responsivity and the incorporation of relevant information in 
decision making is paramount. 
Question 3: Does the current quality assurance plan adequately monitor assessment performance? 

Quality assurance activities are in place but there is some room for improvement pending the 
availability of additional resources. Interviews revealed that staff who are designated to conduct the ITA 
assessments go through a rigorous training led by a seasoned master trainer who has an immense 
amount of experience using the ITA, as well as experience in relevant training activities. Intensive and 
high-quality training is typically the first part of a quality assurance plan for any comprehensive and 
dynamic standardized risk/need assessment tool. Training should include foundational knowledge (e.g., 
the principles of effective intervention, with a focus on the Risk, Need and Responsivity principles and 
how those principles are implemented via high quality assessment activities). Training should also 
include activities that facilitate expertise in the use of the tool, and should go over each item and how 
each item is scored, and how each scored item gets added to a domain’s score as risk or protective. 
Scoring should including scaling (low, medium, high) for each domain as well as overall classification. 

Foundational training, like most learning experiences, should be active and experiential, 
involving numerous realistic and “hands on” examples that require the utilization of newly acquired 
skills. Foundational training should also include at least some initial tests of inter-rater agreement in 
order to insure that all trainees are viewing the same situations similarly and are likewise scoring items 
ina similar manner. Also helpful is the inclusion of training on good interviewing skills, though it is 
recommended that motivational interviewing not be used during ITA assessments. Motivational 
interviewing consists of a defined set of skills and techniques that were designed for assisting the 
interview subject to identify cognitive thinking errors related to their functionality (or lack thereof) and 
other ways in which they might not be living an ideal prosocial life. Motivational interviewing may be a 
useful technique as a pre-treatment intervention (meaning before a youth engages in cognitive 
behavioral group therapy, for example) or as an intervention in and of itself. For the purposes of 
assessment however, the interviewer should make efforts to assess each item and domain as they exist 
currently and in reality. Using motivational interviewing within the context of an objective and 
standardized assessment process might cause the subject to modify their answers if they sense that they 
are offering the “wrong” answer, for example. 

After successfully emerging from initial foundational training, trainees should be afforded a 
period of time during which they can practice using the assessment procedures and conduct scoring in a 


real-world setting, but without influencing interventions for the youth. Audits should occur of these 
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initial practice assessments in order to best insure quality procedures and scoring. As efforts are created 
to insure proficiency, processes should be put in place that indicate when a staff member is ready to 
conduct assessments on their own. 

Once staff have been approved to conduct assessments independently several procedures 
related to quality assurance should be developed. These procedures should include the following, 
tailored to fit the unique needs of JR: 

e Regular tests of inter-rater agreement, and plans for regular tests of inter-rater reliability. 

e Booster sessions on a regular schedule such as an annual conference. Booster sessions would be 
for individuals who successfully completed an initial/foundational training, and would focus on 
some aspect germane to the assessment process (e.g., scoring, reliability, foundational 
principles and vocabulary, case planning, the measurement of change). 

e Quality assurance should include one-on-one observation where a master trainer observes a 
trained staff member conducting a complete assessment, while also conducting their own 
scoring of the individual being assessed. This will serve as an opportunity to offer feedback on 
interviewing skills, overall proficiency, scoring issues, and could also serve as at least a cursory 
test of inter-rater agreement. 

e The ITA should be subject to regularly occurring tests of predictive validity (e.g., every 3 to 5 
years, depending on changes in agency mission, the population of youth, or legislative 
developments that might influence case processing and/or JR resources). 

As noted above, the RAR, RAI, and RACF consist largely of static criminal history items many of 
which are scored automatically from existing data. The same quality assurance efforts should be put 
forth for the items that are not scored automatically and those that are not static in nature. Efforts 
should be made to insure that staff conducting assessments are measuring dynamic items the same 
way, and that there is an acceptable level of interrater agreement. Similarly, efforts should be made to 
standardize potentially subjective information that is currently used to score items. Specifically the 
Client Behavioral Assessment, or CBA as noted above, is a practitioner-driven process that rates the 
youth regarding their behavior. Using real-time/current observational and behavioral-based information 
to score risk/need assessments is potentially very valuable, but also holds the potential to enter 
subjectivity and bias into scoring. As such it is of paramount concern that extensive training and 
definition is afforded to those conducting a CBA. It should also be noted that the CBA may have 
additional utility beyond informing the assessments, such as the measurement of change over time, 


determining when behavioral benchmarks are met, and informing the overall case plan. As noted above, 
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quality assurance efforts as they relate to the RAR, RAI, and RACF should involve regularly scheduled and 
large-scale tests of predictive validity using appropriate outcomes, exploring alternative outcomes, sub- 
populations of the youth (e.g., by race, ethnicity and sex), and should likewise include test for bias. 
Question 4: What recommendations do you have to improve the assessment system in JR, given the 
current level of resources? 

Three overarching recommendations emerge regarding the current assessment system in JR, 
related to four of the assessments currently in use. These recommendations are geared toward 
positioning JR to utilize existing assessments to their fullest, in an effort toward more complete 
implementation of the Risk, Need, and Responsivity principles of effective intervention. 

If the RAR, RAI, and RACF will continue to be used, their use should be limited to other parts of 
the justice system (meaning other than driving a treatment model). Currently these three assessments 
are not fully integrated into the treatment model, nor do they possess characteristics that would 
facilitate a treatment model. These limitations are due in large part to their make-up. The assessments 
are largely made up of static criminal history items, with some dynamic items that may cover a 
criminogenic need, but not to the extent that would be necessary to drive case planning and/or 
intervention. The assessments may have some utility in other areas of the system (meaning other than 
treatment), pending the results of predictive validity tests. As such, their continued use should be based 
in large part on both the results of tests of predictive validity as well as the overall agency objectives. 

Related to the recommendation immediately above, plans should be made for a large-scale and 
current test of predictive validity for all three assessments (RAR, RAI, RACF) using relevant outcomes. 
Under the assumption test results will reveal acceptable levels of predictive validity, these instruments 
should be used for more justice-related (as opposed to treatment-related) decision making. For 
example, the risk assessments might have utility when making decisions about the appropriate level 
(intensity) of community supervision, the appropriate level of residential placement, and other case 
processing decisions related to security (e.g., curfew recommendations, electronic monitoring and the 
like). 

It is recommended that priority is given to the ITA when it comes to driving the treatment model 
currently in place. Of the assessments reviewed the ITA has the greatest potential to contribute to case 
planning, the identification of relevant criminogenic targets, the measurement of change, the overall 
risk/need classification model, and support/evidence when it comes to any number of case processing 
decisions as they relate to a youth’s treatment trajectory. In order to meet the ITA’s full potential it is 


recommended that the instrument is more fully understood across all individuals in the JR system 
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(particularly those that have any contact with youth, abiding by the understanding that any person that 
has contact with a youth can have a role in helping to shape behavior). While the ICI provides an 
overview and summary of the information provided by the ITA, there is benefit to a fuller understanding 
and systemic penetration of the ITA, the information it gathers, and the ways it can be used in order to 
best implement the principles of risk, need and responsivity (though as noted above the ITA may only 
contribute to the assessment of responsivity indirectly). 

Question 5: What additional resources would you recommend to improve the assessment system and 
integration? 

While it is not possible to attach a specific dollar amount in terms of actual additional resources, 
the JR’s assessment system would stand to benefit from one or more of the following 
recommendations. For example, plans should be made to develop a point person who would be 
responsible for all activities and procedures related to assessment within the agency, for example, a 
“Director of Assessment.” The Director of Assessment would likely need at least a small administrative 
structure (staff) that would be dedicated solely to implementing, monitoring, and driving everything 
related to the system’s assessment procedures. The new administrative position would oversee new and 
ongoing training for existing (and newly adopted) assessments. The position would also be responsible 
for conducting quality assurance reviews of case plans derived from assessment activities insuring that 
they are driven by the RNR (Risk Need and Responsivity) principles. Likewise, the position would oversee 
regularly scheduled tests of validity for all assessments in use, as well as efforts to insure acceptable 
levels of interrater agreement and interrater reliability, and would also oversee the adoption and 
implementation of a standardized responsivity assessment process. Under the assumption that 
assessment activities should drive, and be integrated with the treatment models, the Director of 
Assessment would be ultimately responsible for making sure that true integration occurs. Assessment 
procedures are the cornerstone of any effective intervention model, and as such the integrity of the 
information that is gathered, the way the information is used, and the extent to which the information 
undergirds every part of the system is paramount. 

Additional resources would also benefit the expansion of treatment and rehabilitative activities 
within the JR system. As noted above, if additional focus is afforded the ITA, it is likely that gaps in the 
current treatment model will be revealed. These gaps will likely show a need for additional capacity in 
treatment services that can address criminogenic needs in a variety of ways. In addition to revealing the 
need for additional services — meaning both type and capacity (e.g., expanded rehabilitative offerings 


such as cognitive behavioral interventions focused on substance use, relationship building, views of 
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work or school), effective treatment models also are able to vary the intensity and duration of the 
intervention depending on the severity of the criminogenic need. For example, information from the ITA 
may reveal that two youth may require that the same criminogenic need be addressed, but they each 
might need a different amount of programming in terms of intensity and/or duration. In order to allow 
the ITA to drive treatment fully, programming will need to have as much flexibility as possible when 
implementing case plans and addressing treatment targets. 

Additional resources should also be allocated for testing and monitoring the integrity and 
quality of all treatment programming. While this particular recommendation may initially appear to be 
outside of the purview of assessment, credible efforts to assess program quality take the assessment 
model heavily into consideration. Commonly used procedures that quantitatively assess the quality of 
rehabilitative programming will determine whether or not valid assessment procedures are in place, 
how assessment information is gathered and used, and to what extent the assessment information is 
being used to the fullest in driving programming and monitoring both individual progress and program 
effectiveness. In addition program treatment audits consider whether all three principles of risk, need, 
and responsivity are appropriately implemented and are present in every part of the treatment 
program. Resources that are dedicated to program evaluation should be part of a comprehensive 
assessment model. 

Question 6: JR will be serving youth up to age 25 in the future. What advice related to the assessment 
system would you give the agency as they prepare to provide this treatment for older youth? 

It will very likely be necessary to adopt, implement, and test the validity of a tool designed to 
assess the adult offender population. Most if not all of the recommendations mentioned above also 
apply to any adult-based assessment model as well. When comparing the youthful offender population 
to the adult offender population, there is a great deal of overlap regarding the criminogenic needs that 
contribute to an increase in the likelihood that antisocial behavior will occur. However, the form of the 
criminogenic needs will vary, and as such the measurement tools that are used should vary as well. 
Youth for example are more likely to have parental figures as active parts of their lives that in turn 
contribute to the constellation of influences the youth is surrounded by. At the same time, youth are 
less likely to have dependents to consider. Young adults 18 and older are less likely to be influenced by 
parental figures and more likely to have specific factors in need of consideration, such as dependents. In 
addition, any number of status offenses (the use of alcohol and in some states marijuana, for example) 
provides a potentially large degree of difference between youth and adults, and assessment models 


should reflect the difference. Tools such as the LSI-R, the COMPAS, the ORAS, or another comprehensive 
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tool of sufficient breadth and depth that has a high degree of demonstrated predictive validity should be 
considered for the young adult population JR will be serving more of in the future. Once adopted, best 
practices in offender assessment as they relate to training, implementation, and tests of validity on the 
local population should be observed just as those principles that have been recommended above. 

It is possible that portions of the ITA could be used on the young adult offender population, but 
it would be necessary to go to great lengths to ensure that the tool (and/or the portions therein) do 


indeed adequately predict the desired outcomes for young adult offenders aged 18 to 25. 
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Executive Summary 

Washington state Juvenile Rehabilitation (JR) has long been on the forefront of developing and 
implementing Dialectical Behavior Therapy (DBT) — an effort it initially commenced nearly 25 years ago. 
JR has sought to implement DBT to fidelity — across modes and functions. The intention of this review 
was to determine the extent to which DBT is provided at JR and to make recommendations as 

needed. To make this determination, several JR site visits were made during November, 2019. 
Whenever possible, visits included conversation with a range of staff (leadership and frontline youth 
counselors), visiting the units, and speaking with youth. 


Key Findings: 


1. 


The intention to provide DBT to fidelity is deeply rooted — from the behaviorally specific DBT 
standards for each DBT mode to the sophisticated and rich intranet of DBT resources to aid in its 
delivery. Beautifully produced DBT skills training posters line the walls of the units — in plain 
sight for all (youth and counselor) to see/use. When randomly asked, youth are generally able to 
name and describe one or more DBT skills they use. 


Staff shortages and staff turn-over have significantly compromised JR’s ability to train up its 
workforce in DBT and to maintain whatever training gains it makes. Non-competitive wages 
make it particularly challenging to recruit personnel at all levels (AA, BA, MA, PhD). The situation 
is even more dire in rural areas. 


Training, consultation, and supervision resources are insufficient to meet the actual demand 
that counselors have in order to fulfill the DBT standards and deliver DBT to fidelity. In contrast 
to the early years of DBT implementation at JR where staff attended intensive trainings with 
ample consultation from other JR DBT experts, the trainings are now limited to two and three- 
day trainings and very limited consultation by three DBT consultants who span all of Washington 
state. Staff shortages have made it difficult for new staff to attend the trainings that are offered. 


The trainings that are offered are dense in what they seek to cover, include numerous slides. It 
is hard to see how these standard trainings by themselves are sufficient to produce behavior 
change in counselors (let alone adherence to DBT) if not paired with ongoing consultation, 
support, evaluation of their actual performance (recording of sessions for review by experts, 
real-time observations of sessions), opportunities to truly practice and gain feedback, and deep 
engagement with reading/studying other materials. The degree of training made available to the 
original cohort in the late 1990’s was vastly different from what is made available today. This 
early cohort participated in a two-part 10-day intensive training that included reading multiple 
DBT and other treatment manuals + completing a DBT exam (open and closed book) + case 
presentation + program presentation in front of other teams compared to attending traditional 
didactic workshops. The initial group received training directly from DBT treatment developer, 
Marsha M. Linehan, PhD; | co-taught with Dr. Linehan at this initial training. Indeed, the Copalis 
Cottage Treatment Manual was a product of that training. (For what it is worth, there is very 
little evidence to support the efficacy of traditional training workshops to promote behavior 
change in clinical practice). 


The Environmental Adherence (EA) function of DBT has deep roots throughout facilities. The 
DBT standards accurately capture what a gold-standard JR system should seek to do. The DBT 
Skill of the Week, with rich content available through the JR intranet and in a weekly email 
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makes it (in theory) easy to access rich, adherent training and help ensure the teaching of and 
coaching in DBT skills. It appears that most units fulfill the DBT Skills Training standard of 
providing this mode to youth. Fulfilling the standard for DBT consultation team and didactics 
varies significantly by unit and facility. Even in cases where a consultation team meets, it is 
unlikely that the function of this important mode is achieved as members don’t know DBT. With 
respect to the individual delivery of DBT, with the exception of a handful of old-timers who 
received comprehensive training and consultation in DBT, the majority of today’s counselors 
have not have enough training to know how to deliver DBT individual counseling to fidelity. 
Barriers to recording sessions make it impossible to know, but most interviewed felt confident 
that what goes on behind the closed door resembles garden-variety counseling. 


6. Staff are divided over whether it is realistic and necessary to offer comprehensive DBT to all JR 
involved youth and whether it is realistic to expect AA/BA level counselors to provide DBT 
individual counseling to fidelity. We know that it is possible for AA/BA level counselors to deliver 
DBT to fidelity when they are provided the kind of training, supervision, and consultation 
support that is required initially and over a period of time until fidelity (based on review of 
tapes) is achieved. 


Recommendations 


1. Reduce Staff Shortage and Improve Retention. It is difficult to see how to improve system-wide 
fidelity to DBT unless there are enough staff to provide coverage while others attend trainings, 
and that staff stick around after the training investment has been made. 


2. Reconsider Comprehensive DBT for Some (vs. Comprehensive DBT for All). It is a noble effort 
to seek to provide comprehensive DBT for all youth, particularly in light of the real challenges to 
hiring and retention of staff. While DBT for All may be the ideal, it may also not be affordable 
(and therefore realistic). It is better to provide comprehensive DBT well to those who need it the 
most rather DBT Lite for all. Clear criteria could easily be established to determine those who 
most need DBT counseling to fidelity. A small cohort of clinicians/counselors could then be 
trained and supervised to ensure they have the capability of providing DBT to fidelity in the 
individual counseling mode. All other residents could receive DBT Lite (El, DBT skills). Those 
selected to provide 1:1 DBT to fidelity may also have additional incentives to tackle work with 
the most difficult of youth (e.g., more training opportunities, opportunity for more competitive 
wages, protected time for consultation team). 


3. Offer Comprehensive, Intensive Training in DBT over a Year. | would strongly recommend that 
Washington State JR resumes its former practice in sending those counselors that it wishes to 
provide DBT individual counseling to a five-day DBT foundational or 10-day intensive offered by 
a reputable organization, like Behavioral Tech or Portland DBT Institute, Inc. The intensive 
should include not only didactic and role play instruction, but also a DBT exam (open, closed 
book), a requirement to read relevant textbooks (or the Copalis Cottage manual that was 
developed in the late 1990's), individual and team assignments, including presentation of a case 


1 One study by Linehan and colleagues found that it took a PhD level well-trained behavior therapist two years to achieve DBT 
adherence in the individual therapy mode of treatment. This was among a sample of clinicians who also attended the DBT 
intensive and had weekly supervision that included tape review. In my experience, this process can go faster if the clinician is 
embedded within an organization that is steeped in the practice of DBT. For example, at Portland DBT Institute, Inc., it takes 
approximately 12 to 18 months until a licensed clinician is fully adherent in this mode. These clinicians also attend an intensive 
training, complete all its requirements, and submit audio recordings for adherence review. 
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and program to other attendees for feedback. The requirement to share out your work has a 
contingency management function, where people are motivated to do well knowing they will 
need to describe a case/program in front of their peers. In some systems, the five days are 
divided into a set of 3 days (Week 1) and 2 days (Week 2) to reduce the burden in covering work 
for line staff. The existing trainings can augment, but not replace, this comprehensive training. 


Expand the Number of DBT Experts to Provide Support. It appears that the three consultants 
are insufficient to provide the needed coverage to all counselors and program managers 
throughout the JR system. Also, it would be wise that the consultants are also themselves DBT 
experts. One way to ensure that this is the case is by selecting individuals who are Linehan-DBT 
Certified as clinicians. If JR is unable to hire more, it might consider developing a relationship 
with other DBT experts with deep JR experience — either through Behavioral Tech, LLC or 
Portland DBT Institute, Inc. or other comparable group. 


Encourage/Require Audio Recording of Individual Counseling Sessions for Review and 
Feedback. To help counselors learn DBT and to guard against drift, it is important that individual 
counselors record their sessions. These sessions can be reviewed by their supervisors and 
consultant to monitor delivery of treatment and reviewed in consultation team for purposes of 
receiving help. 
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BACKGROUND 


History of DBT in Washington State Juvenile Rehabilitation (JR). Washington State JR began its 
application and implementation of Dialectical Behavior Therapy (DBT) in 1998 when it first began 
developing an innovative adaptation of DBT for its youth in facilities. This highly ambitious adaptation 
was initially developed in a single cottage (Copalis Cottage) on one campus (Echo Glen Children’s Center) 
that served females with mental health problems who also engaged in suicidal and non-suicidal self- 
injurious behaviors. These early adaptations described in the Copalis Cottage manual — a sophisticated 
translation of Linehan’s early treatment manual for Washington State JR staff at all levels. 


As the data from this early work yielded positive findings, DBT was expanded throughout Echo Glen and 
all JR facilities and for the youth it served. During this “golden period”, significant and extensive 
resources were devoted to expert training and consultation in DBT. Indeed, Linehan and/or her protégés 
(Kelly Koerner, PhD, Henry Schmidt, PhD) were among those who provided extensive training and 
consultation to JR over the years. Interviews with staff who recall these early years describe 
extraordinary training/consultation resources: experts listened in on sessions, attended consultation 
teams, and were available for clinical review of complex cases. Low rates of staff turn-over made it 
possible for staff at all levels to routinely attend regular trainings and for these efforts to find their way 
into clinical practice. 


One particularly innovative JR adaptation established early involved delivery of the individual therapy 
mode of DBT. Linehan’s standard outpatient model involved use of masters level and doctoral level 
licensed clinicians to administer individual therapy. The JR model instead relied on unlicensed frontline 
staff with two to four-year college degrees to provide this mode. It rightly reasoned that frontline staff 
have the greatest contact with youth and the capacity to significantly and positively influence 
(behaviorally shape) the youth’s behavior through their contact with youth in the milieu. Presumably, it 
became a cost-efficient way to provide the individual counseling mode, as frontline staff were already 
tasked with meeting with you on a routine basis multiple times a week. The high degree of training, 
consultation, and supervision, in combination with the system’s overall investment in DBT 
implementation, made it possible to provide the requisite training to individuals not formally trained as 
clinicians. One benefit of this approach was that DBT principles, strategies and philosophy could be 
woven into the youths’ care from the time they woke in the morning to the time they sent to bed. For 
example, youth in Copalis began their day with mindfulness, goals setting, and DBT skills they could use 
to achieve their daily goals. To ensure DBT was as much of a voluntary choice as possible, counselors 
would give youth the choice of whether to participate in the morning group. Those who opted to 
participate were first in line for breakfast (contingency management strategy). 


Within a decade of its initial effort, Washington state’s JR DBT adaptation became the gold-standard for 
other similar facilities. Indeed, the “Echo Glen” model was disseminated to other juvenile justice 
facilities throughout the United States and other parts of the world. Its “home-grown” DBT experts 
(those who were at its inception), including Dr. Schmidt, Brad Beach, Ron Baltrusus, grew over time a 
robust in-house training team. Under the stewardship of Ted Ryle and his colleagues, JR has developed a 
top-notch set of DBT training materials and processes that is the envy of other large system 
implementations and other professional DBT training organizations. These sophisticated training and 
clinical materials remain readily available to all JR staff via their well-developed intranet. 


Findings from An Evaluation of Dialectical Behavior Therapy in Washington State’s Juvenile 
Rehabilitation (2019). While numerous studies had been conducted on its outcomes, no formal 
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processes had been implemented to measure the quality or adherence of individual counseling sessions, 
skills groups, or consultation team from 2000 to 2019. Quality Assurance (QA) protocol only monitored 
environmental adherence (EA; one of four DBT treatment modes known as “structuring the 
environment” to enable delivery of DBT throughout the treatment environment). The results of this 
undertaking are summarized in this 2019 report. Key findings detailed in the report include: 


e = In 2014 and 2015, about 57% of youth who left JR did not receive the level of treatment 
required by at least one of the three separate standards (EA, individual sessions, and skills 


groups) 


e About 21% of youth received treatment that met one standard, 18% received treatment that 
met two standards. Only 4% of youth left JR having received treatment that met all three 
standards. 


e Increases in EA were associated with reductions in felony recidivism. 


e Rates of individual sessions and skills group were not related to recidivism, however, the quality 
of those sessions varied widely and was not systematically assessed. 


e = High levels of EA were particularly important for younger youth and those with a high risk of 
mental health issues. 


On the basis of these findings, recommendations included: 
e To begin monitoring individual session, skills training and consultation teams 


e JRshould prioritize high-level implementation in the acute mental health units. Training and 
staff resources should be dedicated to these units to bring them up to standard as quickly as 
possible. 


ASSESSMENT 


Materials Review. In advance of visits and interviews at facilities, | was provided ample examples from 
the JR Home Page/SharePoint as well as the PowerPoint curricula for three DBT trainings. (Please see 
Attachment A for a summary and review of these materials). Washington State JR has provided its 
counselors with comprehensive, DBT adherent materials to enable the standardized delivery of DBT 
skills through its impressive Skill of the Week materials. This includes a weekly email, a carefully 
constructed indexing of skills, easy to access documents to support training that are relevant to their 
population. Anyone with access to these materials can easily download a full teaching curriculum for 
DBT skills training with minimum effort. The two-day workshop materials are both adherent and 
exhaustive in their review of DBT. (Indeed, my only concern is that there is too much content to deliver 
in two days, particularly if demonstration and role plays are involved. No doubt JR trainers have 
considered this and know how best to tailor the content to their audience). 


OBSERVATION 


Green Hill School (GHS). | visited GHS on two occasions: November 8, 2019 and November 12, 2019. The 
primary purpose of the second visit was to visit the units and speak directly with as many line staff and 
youth, as possible. On both visits, staff spoke candidly about their experiences at GHS. It was clear that 
staff cared about their work, helping their youth, and supporting each other. It was immediately 
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apparent to me that all staff had a shared behavioral (DBT) framework for their work. Notable 
comments included: 


Concern that the size of the three larger units (Maple, Spruce, and Hawthorn) make it difficult 
for staff to provide the expected level of therapeutic intervention. If one youth is on an 
observation level (comparable to a one-on-one in an inpatient hospital setting) as part of the 
system’s suicide prevention procedure, for example, it is very difficult to leave the floor to have 
a counseling session with another youth. The same number of staff are assigned to the large 
units as to the small units. Those staff with 40 youth per unit have “no time to deal with things 
that arise in the moment”. 


Staff Shortages, Staff Hiring and Staff Turn-Over Challenges. Staff turn-over is high with an 
annual staff turn-over rate of approximately 30%. Most staff are in their first two years of 
employment and lack training. When asked about the high rate of turn-over, the supervisor | 
spoke with noted that the Washington State JR salaries are no longer competitive; Big Box 
stores, like Walmart, pay comparable salaries and offer comparable benefits for a much less 
stressful job. He stated while hiring has improved, the incentives to retain staff (salary and 
benefits) are not competitive, which is likely then to further perpetuate the problem even if 
hiring efforts are improved. 


Insufficient Resources to Train New Staff. Because of the shortages, there is simply insufficient 
coverage to allow new (and less new) staff to receive training in DBT. | met with one supervisor 
who shared with me his log of trainings attended by his staff: most of the newer staff (and some 
of the less new staff) had not attended basic DBT trainings. He simply could not free them up 
enough to attend the required training. Staff shortages and limited “off the floor” time also 
mean that counselors do not have time to use the JR Homepage to augment their training. 


The most well-trained employees work on the day shifts while the youth are in school. The 
newest and least trained staff work the swing shift and while they are doing their best, they are 
more likely to end up reinforcing dysfunctional behavior. 


Staff Shortages Compromise Delivery of DBT. Staff shortages also make it difficult to make time 
to provide weekly counseling sessions. In theory, each counselor is assigned between two and 
six youth to meet with during their day they are scheduled to be “off the floor”. Seldom are 
there sufficient staff available to allow counselors to have their scheduled “off the floor” shift. 
There was a general belief among those | spoke with that while JR says DBT is important, there is 
little investment of resources to make successful implementation possible. There simply are not 
enough staff to provide coverage for all shifts to allow new employees to attend trainings. The 
trainings that receive highest priority are those required to onboard new employees to safety 
procedures. For example, a 40-hour course in defensive tactics is required by all new hires. 
While DBT courses are available, they are lower in priority because of staffing shortages. 


It is estimated that only 20% (max) of counselors attend consultation team. Among staff | spoke 
with, some did not know that consultation teams were provided. Many are simply unable to 
leave the floor due to staff shortages to attend a DBT team meeting. 


Variability in Quality of Care. Youth assigned to counselors who have been in JR for many years 
have a fundamentally different experience and care than those who are within their first five 
years at GHS because of the staff shortages and limited ability to take part in training. Many 
counselors are unable to get off the floor to provide individual counseling sessions with their 
youth. When speaking with supervisors and managers about the quality of individual therapy, 
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the majority of those | spoke with indicated that they have little reason to believe that youth are 
receiving DBT counseling during their 1:1 appointments because they are insufficiently trained. 
When asked what treatment was provided, those | interviewed suggested it was most likely 
garden-variety counseling that they learned during their undergraduate college training. 
Because sessions are neither recorded or observed, it is impossible to actually know what 
occurs. With respect to delivery of DBT skills training groups, many counselors simply lack the 
ability and training to know how to conduct the DBT skills training group. While there is a 
commitment (DBT standard) to have all youth attend a DBT skills training session per week, it 
does not reliably occur. 


One supervisor | interviewed described a very elaborate token economy he implemented on his 
unit. After some discussion of its merits and the system’s willingness to provide him money to 
purchase items his youth would work for, he acknowledged that it was inconsistently used 
because his staff were not sufficiently trained in behavioral principles to use it well, and that 
staff shortages also interfered with consistent delivery of tokens and rewards. 


Professionally Developed Posters. The units are lined with professionally produced posters of 
the DBT skills at a height that makes it difficult/impossible to get torn down. 


Naselle Youth Camp. | visited Naselle on November 25, 2019 for approximately four hours, during which 
time | had an opportunity to speak to its leadership team, youth counselors, view its school and several 
of its units. | was unable to speak directly with any youth because the hours of my visit coincided with 
their school day. The staff were generous in their time meeting with me and in their candor. As was the 
case at GHS, it was abundantly clear that those | spoke with were passionate about their jobs, serving 
their youth, and providing the highest quality clinical care. Notable highlights from the conversation 
included: 


Staff Hiring and Retention Challenges. Because of Naselle’s location, it is far more difficult for 
them to hire as the pool of available applicants is small. Average non-competitive salaries 
further complicate hiring and retention of staff. Those that have stayed on for decades often 
have a corrections mindset with old biases that are inconsistent with JR’s current treatment- 
focused approach. Many who were initially trained during the “golden years” have left. They 
highlighted that their staff psychologist position, for example, has been open for some time, 
further leading them to depend on the already stretched too thin JR DBT Consultants. 


Insufficient Resources to Train New Staff. Similar to feedback from GHS personnel, those | 
interviewed at Naselle stated that there is significantly fewer training, consultation, and 
supervision resources available to them now compared to two decades ago when DBT was 
rolling out. Resources included: greater available time to train and having easy to access DBT 
consultants who would come into join consultation team to help develop a case 
conceptualization for a particularly difficult youth. 


During a meeting with program managers, one individual indicated that while new staff are 
supposed to complete a three-day DBT training early in their employment, most of their staff 
had been unable to attend the training but instead learn as best they can by watching peers or 
through self-study. 


A number of staff pointed out that the DBT consultants do a fine job, but they are spread too 


Portland DBT 
a INSTITUTE 


thin to be of real, enduring benefit. “When (consultant) is here, (they) does a great job helping 
us with a difficult kid. But she is stretched thin.” 


Youth are More Complex. Youth who are coming into JR facilities are far more complex than 
they were even several years ago and trending to become more complex still. This speaks 
volumes to the importance of doing DBT well. 


Variability in Quality of Care. There is a significant disparity in care received by youth working 
with a more senior counselor and those assigned to a “rookie”. The expectation that youth will 
receive DBT from Day 1 is unrealistic because new staff are not trained and getting them trained 
up in light of staff shortages that prevent them from attending trainings further complicate 
matters. 


“DBT Lite at Best”. Because of staff shortages and significant reductions in training, supervision, 
consultation support to DO DBT, the overall consensus was that youth throughout the facility all 
receive some DBT, but that it is “DBT Lite” — consisting of “rudimentary BCAs and maybe a skill 
or two to help the youth”. Naselle relies on a lot of “blocking” — structuring up the environment 
as a whole to limit/prevent dysfunctional behavior versus personalized youth-centered 
treatment plans. 


No Quality Control over Counseling Session. Leadership acknowledged that they have little idea 
what occurs in individual therapy sessions because of the lack of tape review. Some of this is 
due to a perception that unions will push back on JR requiring counselors to record their 
sessions. 


Problematic Nature of Counseling + Corrections Mode. Several questioned the original wisdom 
of the decision to have AA and BA-level counselors deliver the therapy — both because they lack 
the formal training and because of the incompatible roles they play with the same youth in 
terms of what is required while on the floor vs. in a therapy session. Essentially, there is a dual 
role between being security guard and clinician. 


Who Needs Therapy? Who is Qualified to be a Counselor? There was considerable conversation 
about whether every youth needed a highly specialized DBT clinician and/or comprehensive 
DBT. In their view, about 75% of the patient population will be positively impacted by the 
therapeutic environment alone: having established structure, routines, consistent use of rules, 
rewards, punishments. However, there is another 25% of youth who have severe and complex 
problems and require more than the therapeutic milieu. These youth, in their view, are best 
served by having comprehensive DBT delivered to fidelity and by a qualified (and licensed) 
clinicians. Their view was rather than trying to make comprehensive DBT available to all (since it 
is not possible given the resources), focus DBT on only those who need it the most — consistent 
with the original roll out of DBT at Echo Glen. In this model, develop specific criteria for those 
youth who need comprehensive DBT. One person recommended that there may still be a role 
for the youth counselor, but more as a case manager than thinking they are providing 1:1 
individual therapy. 


In recounting the decisions that led to the comprehensive system-wide roll out of DBT and 1:1 
therapy provided by a BA/AA-level youth counselor, some staff recalled it being a point of 
contention from the start. 


Coaching on the Floor: They estimate that between 50% to 85% of their staff know how to 
coach a youth “on the floor”. 
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- Consultation Team: “We have a if you are drowning, we can help” approach now versus a 
planful approach. System tries to ensure that its counselors receive weekly consultation team. 
The first hour is devoted to consultation; the second to administrative issues. Staff use a paper 
outline for team that has been handed down over the years. It does not include didactic training 
during this time. Team is also compromised by radios, need for 911 response to problems. 


- Skills Training Groups: Occur weekly for youth, but often have 14 youth in a group at a time. 
(Difficult to engage youth with this size). 


- Overall Exposure to DBT: Described as sporadic. Leadership estimates that only about 25% of 
youth receive the DBT model, as intended by JR. “We do really good at presenting on paper 
what we do, but it doesn’t match what we actually do”. 


Touchstone. | met CF administrators. Both highlighted that when DBT first began, there was “tons of 
support for the (DBT) model — all the way to the top.” Yolanda recalled the transition from a punishment 
model (in the 1990s) to use of more therapeutic interventions, including DBT. Because the work became 
harder over time, upper leadership received a lot of pushback from employees and the union, but 
leadership — up to the Secretary of the agency — were all aligned on the transition to a therapeutic 
approach. Managers felt highest levels of leadership “had our backs” as they implemented the difficult 
change. Ample training resource was provided to help support the significant culture change. Over the 
years, as fewer resources are available to support the implementation of DBT, the abundance of support 
has been replaced with a “every man for himself” mentality. The roll out from Echo Glen to the system 
at large required considerably more help and resource than what was provided. The best laid plans — 
development of standards for delivery of DBT to fidelity - are compromised by staff shortages and staff 
turn-overs. A number of the most well-trained staff left JR and joined other outside organizations, 
including Linehan’s training arm. 


- Insufficient Training: They highlight that at the start of implementing DBT, staff went to 16 to 32 
hours of training at one time. But now, they have a handful of trainings (coaching on the floor 
and case management trainings were named), but that is it. As managers, they both find it 
difficult to hold staff accountable for not doing DBT if they have not been trained in the model. 
While some trainings exist, they simply do not have the staff sufficient to provide coverage. 


- Consultation Team/Didactic: They follow the standard of dedicating one-hour weekly to each 
but “nine times out of 10, people have to leave to transport a kid”. The quality of consultation 
team is also compromised by the lack of training in DBT: “How do you do true consultation 
without knowing DBT?” 


- Variability In DBT Delivery by Site. In some locations with well trained staff (those trained in the 
early days of DBT, “(their) girls are getting DBT to death”. But other locations with newer staff 
don’t have the same resource. Youth are receiving DBT skills mode as set forth in the standards. 
Yolanda is new to Touchstone where the staff is not trained and where there have been higher 
rates of turn over. There, youth are not (yet) receiving DBT. 


- Recommendations for Improvement. Personnel felt that it is imperative to reduce staff turn 
over and increase training in order to fulfill the JR standards. That may begin with a significant 
wage increase to compete with other less stressful jobs that pay comparable wages. 
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ECHO GLEN CHILDREN’S CENTER: | met with senior officials on November 27, 2019 at Echo Glen. At the 
time of that report, six of eight cottages were providing ITM Case Management/Counseling as specified 
in the standards. In the two that didn’t meet the minimum standard, they are not far off (approximately 
70% in those two settings). With respect to group, the standard is met in 60% of the cases, which may 
be due to a failure to document actual groups that are occurring. Groups however were described as 
“not adherent to DBT...we don’t really know how adherent they are”. With respect to consultation team, 
each manager at Echo Glen is required to run a consultation team, but realistically, “a lot of things 
compete for the time” including reviews for youth. She estimates that 50% of the units are conducting 
consultation team 80% or more of the time, but that they are not adherent to DBT. With respect to case 
management, personnel highlighted that with staff turn-over, they have fallen “way off’: “New people 
coming in just don’t have training before they can get it. Staff that have had training are tired, burned 
out and cut corners”. In comparison to comparable services at other facilities, Echo Glen “scores off the 
charts” on fulfilling the specific behavioral targets for environmental adherence. 


| asked interviewees in their view contributed to Echo Glen’s overall success at rolling out DBT. They 
highlighted the following factors: 


- They began a pilot project with the most challenging cottage that had the poorest outcomes. 
Rather than rolling it out to the whole system, they started small and demonstrated positive 
outcomes that caused other cottages and leadership to take notice. 

- They began with a high degree of buy in from top echelons of leadership. The system’s 
commitment to DBT fidelity was demonstrated through “tons of training and support”. 

- Echo Glen slowly moved DBT to other cottages after Copalis outcomes provided a proof point. 

- Asthey systematically rolled DBT out to other cottages within Echo Glen, those cottages had the 
ample DBT expertise (Dr. Henry Schmidt and Brad Beach) onsite to provide immediate, in the 
moment coaching and support. 


Several current challenges make it now difficult to replicate these earlier conditions: 


- Staff turn-over has been particularly hard the past two years: “It’s so hard to keep staff around 
here”. 

- | System does not provide Echo Glen or other facilities with the level of training, supervision, 
consultation, and support to keep staff adherent, as well as motivated to do DBT. In total, the 
system has three DBT consultants, only one of whom is an certified by the DBT-Linehan Board of 
Certification. No onsite expertise is available for in-the-moment consultation of difficult 
situations. (One system leader is required to fill in for what Diana Frey and Henry Schmidt used 
to do). 

- The push to do DBT is no longer collaborative but is now top down. (Please note that this is my 
interpretation of a more complex nuanced discussion). 


CONSULTANT INTERVIEW. | spoke with consultant on Friday, November 29, 2019. | reviewed my 
understanding of what | had learned to date. She commented, “That is spot on.” She also described her 
and the team’s approach to training and emphasized that there is more need for training and 
consultation than she and her colleagues can themselves fulfill. She is also clearly passionate about her 
work, cares deeply about supporting JR staff, does her absolute best to develop and deliver new training 
content to address their needs, and is highly competent and skilled at both doing DBT and training JR 
staff in DBT. We also discussed several specific questions she had for me. 
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OTHER OVERALL COMMENTS/THEMES: Several themes emerged repeatedly by people at all ranks and 
at every facility | spoke with. These themes are described below: 


Problem of Staff Turn Over: Many had observed that recent leadership “didn’t sound the alarm fast 
enough” to address the problem. Until recently, wages were described as “deplorable” for the demands 
placed on them, the transportation burden (most facilities are in rural locations without easy access to 
public transportation), and the fact that the youth coming into program were increasingly more 
complex. A recent 8% wage increase has helped, but not fully solved the problem. The benefits and 
other “perks” that made employment by JR an easy choice for many are now gone, but are not replaced 
with higher salaries (wanted by many younger employees in the labor market today). 


Staff Burn Out. As many well-trained people have left or retired, it has placed greater stress and strain 
on those who are left — to work harder and assume more responsibility. These efforts are overlooked by 
leadership. Additionally, the backbone of service delivery at JR (counselors) were not included in the 
most recent salary negotiation, thus compromising their own morale. 


Solution for Training: Universally, people pointed to staff shortage as being one of the biggest factors 
that interferes, but not the only factor. The overall investment in training, consultation, and supervision 
is insufficient for the volume of counselors expected to provide DBT. Additionally, of the three DBT 
consultants, only one JR consultation staff is perceived universally as a DBT expert in doing and teaching 
DBT. She is the only one that is, for example, certified by the Linehan-DBT Board of Certification. While 
the online materials are available, staff either do not have time to access them or are not motivated to 
do so. 
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APPENDIX A: QUESTION INFORMING REPORT 


Washington State JR provided the following questions for consideration during my visits and to guide 
the summary of my findings. 


1. Are youth being appropriately matched with treatment based on need? 
a. What assessment is being used for program eligibility? 
b. Is the assessment being used appropriately? 
You are not being appropriately matched to the treatment model. The current model is that all 
youth will receive comprehensive DBT. The system however is ill-equipped to deliver on its promise 
because of staff shortages. As a result, few if any youth receive comprehensive DBT. Please see 
recommendations. Unless the system wishes to invest significant financial resources to train its 
workforce in DBT to fidelity (in all modes), a better approach would be to provide comprehensive 
DBT to those with the greatest need — either those with the most severe and complex emotion- 
based mental health problem and/or those who do not show progress receiving the standard care 
as usual. 


2. Is the treatment high quality? 
a. How well is JR implementing this treatment? 
b. Are youth receiving the appropriate dosage? 
c. Do staff receive the correct type and amount of training? 
d. Does there seem to be variation by location? (between institutions, between institutions and community 
facilities) 
Please see attached report. 


3. How well is the treatment integrated with the assessments, other treatments, and reentry 
planning? 
While DBT is well integrated environmentally and with other treatments provided, there is no 
means of determining which patients are more complex and difficult to treat and need more full- 
fidelity DBT. As noted above, it is simply not feasible that full-fidelity DBT (to adherence) is delivered 
to every youth in this context of severe staff shortages and high rates of staff turn-over, combined 
with limited and sustained comprehensive training. 


4. Does the current quality assurance plan adequately measure the treatment model? 
More emphasis is required for recording of actual counseling sessions. 


5. Do you see any evidence that would concern you that treatment access varies by race, ethnicity, 
age, or gender? 
No evidence. 


6. What recommendations do you have to improve implementation given the current level of 
resources? 
Please see recommendations. 


7. What additional resources would you recommend to improve treatment quality and integration? 
Please see recommendations. 


8. JR will be serving youth up to age 25 in the future. What advice would you give the agency as they 
prepare to provide this treatment for older youth? 
Recommenactions will help ensure overall excellence of clinical care, including those who are older. 
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Appendix B: Materials Reviewed 


1. DBT Standards 


a. 


DBT Standard 1: Individual Counseling Sessions for DBT and ITM. Sessions are to be 
provided no less than one time weekly. Specific counseling tasks are delineated for two 
treatment phases: pre-treatment phase and treatment phase. Specific tasks for each 
phase are adherent with standard DBT therapy sessions. Each phase includes facilitating 
youth’s engagement in treatment and establishing rapport with youth. The treatment 
phase is fully consistent with the structural tasks required of an adherent/competent 
therapy session. Those who access this standard from within the JR firewall are able to 
access a Pre-Treatment Session Guide and a Treatment Session Guide. (| was unable to 
access these documents). 


DBT Standard 2: Consultation Team and Didactics. Consistent with this mode of 
treatment as specified by Linehan, the function of this mode is to enhance the capability 
and motivation of those engaged in youth care. Consultation team is described to 
enhance case-carrying counselors’ ability to conduct treatment and build collaboration 
around treatment delivery. It is an expectation that those who have direct client contact 
will “review and comply” with this standard. Consultation teams are to meet at least 
once every other week and are to be led by a DBT consultant at least once monthly. 
Didactics are to also be provided at least every other week for case-carrying counselors 
and facilitated by trained staff and/or the DBT consultants. 


DBT Standard 3: Milieu Management. This standard specifies the environmental, 
milieu-based procedures used to support the youth with behavior change and progress 
with program goals, and in their transition to the community. It includes daily routines, 
treatment groups, use of consistent rules, and enjoyable activities. Three levels of 
reinforcement are specified for the milieu: material (token economy), 
environmental/relationship (peer support), and intrinsic (self-motivation). Staff are 
instructed to interact with and coach youth based on their individual developmental and 
skills level, culture, and current emotional state; and to work with youth to generalize 
their work from individual and group sessions to the milieu. This includes “coaching on 
the fly” strategies and “consultation to youth” (vs. environmental intervention) 
approaches. 


DBT Standard 4: Skills Groups. This standard for skills training indicates that all JR youth 
should be attend group at least once weekly and that a consistent structure is used to 
conduct the groups. This structure is fully adherent to DBT. Groups are divided between 
skills acquisition and strengthening. (The document indicates that the focus is on skills 
acquisition and generalization, but by definition, this is more likely strengthening and 
not generalization). All modules of DBT are incorporated, including Middle Path — an 
adaptation for adolescents by Miller, Rathus, and Linehan. The standard recommends 
that each group is taught by two staff, but allows for a single facilitator when two staff 
are unable. 


Integrated Treatment Model (ITM) Standard 2: Supervision. The focus of this standard 
is the supervision of rehabilitation-focused case management and counseling activities. 
Supervisors are required by this standard to provide once monthly supervision to all 
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direct reports. The scope of this standard is beyond the supervision, monitoring, and/or 
adherence of DBT, but does include require that supervisors ensure their reports are 
properly onboarded to the JR expectations for delivery of DBT and other evidence-based 
approaches, and that their reports engage in the required training. Performance reviews 
also include an ITM focus. 


2. JR DBT Training Materials: 


a. 


Milieu Treatment - DBT Coaching on the Floor. This two-day training is comprised of a 
total of 93 slides and focuses on use of behavioral principles and strategies to support 
skillful behaviors in youth and successful reentry. Topics include the DBT’s biosocial 
theory, DBT assumptions, validation levels and strategies, modes and functions of DBT 
treatment, review of behavioral principles, mindfulness skills, distress tolerance skills, 
interpersonal effectiveness skills, motivation and commitment strategies, STOP and TIPP 
skills, problem-solving strategies, and opportunities to practice. The sides are thorough 
and accurate. 


Case Management: Reducing Risk Factors and Shaping Skillful Behavior with High Risk 
Youth (Winter, 2019). This comprehensive 150-slide training begins with an overview of 
working with high-risk offenders, then breaks down treatment tasks required in pre- 
treatment phase and treatment phase of DBT. Pre-treatment includes tasks required by 
counselor in advance of meeting with youth (review of their history, offense) as well as 
tasks required to complete in the first two to four sessions. Emphasis is placed on how 
to engage youth (empathy, validation) as well as topics the counselor must address in 
the first four sessions. This includes a behavioral chain analysis of their committing 
offense, commitment strategies, and goal setting to have a life worth living. To ensure 
motivation by youth to engage in treatment, linking the youth’s goals to treatment tasks 
is also emphasized. Trainees learn details of a behavioral chain analysis and are provided 
opportunities to observe a demonstration and practice. Individual counseling session 
therapy tasks are reviewed. These include: targeting, use of the diary card, and building 
a shaping plan. Day 2 focuses on the treatment phase, including the structure of the 
session, use of behavioral chain analyses, skills coaching, change strategies (contingency 
management, cognitive modification, exposure, skills training), how to respond to 
dysfunctional behavior that occurs in session, and RTM meeting purpose/agenda. 
Throughout this training, attendees also learn how to document their findings and 
content from sessions. (As an outsider, it seems that this is a lot of content to cover in 
two days, and seems more consistent with what could be done in twice the time). DBT 
content is accurate. 


Teaching DBT Skills to Support Skillful Behavior and Successful Retry (Summer 2019). 
This training focuses on learning all the DBT skills, how to structure up the DBT group, 
and the standards for skills training. Discussion is also provided for how to manage 
difficult moments in group. While a small point, the concept of skills strengthening and 
skills generalization seem conflated. A fantastic cognitive map of all the DBT skills is 
included. Only one slide is devoted to emotion regulation. 


3. Examples of Additional Training Support: A series of files were provided by Ted Ryle and 
reviewed as part of this evaluation. They include: 
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Steps for Leading a Mindfulness Practice. This content is fully consistent with training 
materials developed by Linehan’s training organization, Behavioral Tech, LLC and 
includes helpful, specific examples for youth. The content is fully accurate. 


Skill of the Week Index. This two-page document indexes the application of JR’s Skill of 
the Week program. It represents a comprehensive review of all DBT skills. 


How to Navigate the Skill of the Week. This seven-page document provides a thorough 
guide (text and screenshots) to accessing the plethora of JR materials for their 25-week 
skill of the week curriculum available through the JR Homepage (JR SharePoint). A 
month-at-a glance calendar displays the skill of the week. Clicking on the calendar event 
brings the user to a description of content long with PDFs of associated skills curriculum 
documents. A legend allows viewer to see what’s new or what’s been recently 
improved. 


How to Use DBT Skills Facilitator Notes. This seven-page document is a guide for using 
another important resource: the DBT Skills Facilitator Notes. JR has essentially pulled 
together content — basic and advanced — for the teaching/facilitation of each group. 
Content is culled from the two gold-standard DBT skills training manuals: Linehan’s 
second edition revision and the Rathus/Miller skills training manual. Clear and accurate 
instructions are provided for how to structure the skills training session, from soup to 
nuts. A listing of DBT consultants is also provided for assistance if needed (by facility). 


DBT Skills Facilitator Note: Week 1 Mindfulness. This is an example of the rich, DBT 
adherent materials skills trainers receive in order to teach DBT skills group easily and 
simply, without a lot of prep. It goes through exactly what to do and what to say, discuss 
for the topic — for the entire group. It includes suggested activities, how to orient the 
youth to the class today, the main points to be discussed, handouts, activities, and 
recommended homework. What’s fantastic is that the content is pulled from multiple 
sources: Linehan, Miller and Rathus. 


Mindfulness Activities. This 38-page document is filled with very specific mindfulness 
activities appropriate for youth. Each includes a description of how to do the activity, 
the specific mindfulness skill the activity emphasizes, and instructions for debriefing the 
practice. This content is fantastic, thorough, accurate, and would make a great 
publication for others working with this population. This represents an example of the 
supporting skills documents that are available within the JR Homepage. 


How to do Behavioral Rehearsal. This incredibly important accurate two-page 
document summarizes the steps in doing behavior rehearsal well. This document allows 
the counselor to conduct behavioral rehearsal in a group context. 


Weekly Email of DBT Skill of the Week. In addition to the incredibly rich JR Homepage, 
counselors receive a weekly email describing the skill of the week in detail. Illustrations 
and real-life, relevant examples are provided to increase the odds that viewers will pay 
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attention to and review the document. To make it easy, a number of links are 
embedded to relevant SharePoint documents. 


DBT Skills Flow Chart. This single slide is a beautiful, accurate, and easy to follow 
cognitive map of all the DBT skills. It’s a very helpful tool for people just learning the 
skills for the first time. 
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APPENDIX C: QUALIFICATIONS OF REVIEWER 


Linda A. Dimeff, PhD, is Chief Scientific Officer at the Evidence-Based Practice Institute, Inc; Institute 
Director at Portland DBT Institute, Inc., and Clinical Faculty in the Department of Psychology at the 
University of Washington. Since 1994, Dr. Dimeff has collaborated closely with Marsha M. Linehan to 
develop and evaluate an adaptation of DBT for substance-dependent individuals with borderline 
personality disorder; to produce DBT training materials for clinicians; and to train, consult, and supervise 
clinicians in their practice of DBT. She has worked with public and private sector systems throughout the 
world in their efforts to implement DBT. Dr. Dimeff is a recipient of the Cindy J. Sanderson Outstanding 
Educator Award from the International Society for the Improvement and Teaching of DBT. Linda has 
received over 20 federal grants to facilitate the dissemination of evidence-based therapies and has 
published over 55 peer-reviewed publications. She is also the first author of Brief Alcohol Screening and 
Intervention College Students. 


Appendix E: Implementation Assessment of Functional Family Therapy and Functional 
Family Parole 
by Dr. Holly Barrett Waldron 


REPORT TO THE LEGISLATURE 


Fidelity Assessment of the Integrated Treatment Model (ITM): 
Functional Family Therapy (FFT) and Functional Family Parole (FFP) 


Holly Barrett Waldron, Ph.D. 
March 16, 2020 


The Washington State Legislature designated funding for a fidelity assessment of the 
Integrated Treatment Model (ITM) to inform the implementation of services the 
Washington State Department of Children, Youth & Families (DCYF) provides to youth in 
the Juvenile Rehabilitation Parole (JRP) program. This report addresses the 
implementation of Functional Family Therapy (FFT) and Functional Family Parole (FFP) 
services delivered within JRP. The assessment of FFT and FFP implementation involved 
site visits to three geographically diverse JRP programs, interviews with administrators, 
program managers, and staff, and a review of materials (training materials, reports, 
research studies, documents specifying policies and procedures for FFT and FFP 
implementation). Areas addressed in evaluating FFT and FFP implementation include 
how well youth are matched to FFT and FFP services, the quality of treatment services, 
the integration of treatment services with other JRP and external services, quality 
assurance monitoring, and potential variations in FFT and FFP service delivery for 
subgroups of youth. The report concludes with recommendations for improvements 
achievable within the current available resources and other recommendations for needed 
improvements that require additional funding to accomplish. The qualifications of the 
author of this report are also provided. 


Matching Youth with Treatment 


The fidelity assessment of the Integrated Treatment Model (ITM) for Functional Family 
Therapy (FFT) and Functional Family Parole (FFP) services includes an examination of 
the extent to which youth in JRP are being appropriately matched with treatment based 
on need. The matching process incorporates a consideration of the assessment used to 
determine program eligibility and the appropriateness of the assessment implementation. 


In addressing these issues, the determination of the State of Washington legislature that 
half of all incarcerated youth will receive services through JRP while the other half will not 
is a critical factor. This determination presumes that youth re-entering the community 
either have a need for services or they do not have a need for services and that these 
two groups of youth are orthogonal and non-overlapping. This may be an efficient way of 
allocating limited resources, but is not a valid approach for matching youth with needed 
services. 


Currently, 50% of youth eligible for parole are placed in a JRP program. This placement 
includes all youth who committed an offense that is legislatively mandated to include 
parole and youth eligible for parole who score in the top 25% of an actuarial risk 
assessment. The risk assessment scores guide the placement, although staff have some 
flexibility to place some youth in JRP who would not have been included based on their 
assessment scores alone because of a perceived risk due to family environment, 
behavioral health risks, or other factors. This flexibility is intended to ensure that youth 
most in need of JRP services receive them. Unfortunately, the risk assessment is an 
internally developed instrument that has not been validated. Without validation, the 
accuracy of placements is unknown and significant numbers of youth may be 
underserved due to scores that result in no JRP placement upon reentry into their 
communities, while many other youths at lower risk may be placed in JRP. More 
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importantly, any validated assessment tool — even an assessment that would provide a 
finer grained measure of youth functioning across a number of domains (i.e., severity of 
behavioral problems, mental health functioning, family and peer relations, and 
school/occupational needs prior to youth re-entry) would likely have serious limitations in 
determining which youth would benefit most from JRP services. 


Overall, any risk assessment and ultimate placement decisions likely account for a small 
to moderate amount of the variance in identifying youths who should received JRP 
services, as there is likely considerable overlap across youth placed and not placed in 
JRP. That is, the assessments scores for all youth released from incarceration occur 
along a continuum without a known, definitive threshold for predicting which youth are in 
need of or will benefit from JRP services. The placement of 50% of re-entry youth in JRP, 
with the other 50% not placed, is arbitrary and likely produces potentially overlapping bell- 
shaped distributions of need for services among youth who do and who do not receive 
services through JRP. Prior studies have suggested that nearly half of all youth who 
released with no parole aftercare services have identified mental health needs and the 
vast majority of these needs go unmet (Belenko et al., 2017; Washington State DSHS 
Report, 2017; Zajac et al., 2015). It is likely that a significant number of non-placed youth 
are in need of monitoring and have significant reentry needs for services that would be 
best provided by JRP programs. 


Monitoring and providing services to non-placed youth would likely result in substantial 
savings to the state through decreased recidivism and re-incarceration (see meta-analytic 
studies of juvenile reentry intensive aftercare: Bouchard & Wong, 2018; James et al., 
2013; see also FFP studies: Lucenko et al., 2011; Risk, 2009; Sexton et al., 2013). Thus, 
because many youth who are not placed in JRP will recidivate and experience significant 
problems after reentry, it cannot be concluded that youth are wholly appropriately 
matched with services. Although the current assessment protocols appear to be 
implemented with integrity and the assessment process seems to be a critical guide in 
determining how services such as FFT, wrap-around services, job assistance, housing, 
and transportation, within JRP would be allocated, the use of the assessment approach 
for determining who receives JRP services and who does not is an inappropriate and 
fiscally inefficient strategy for ensuring the positive transition of youth during the reentry 
and post-reentry period. 


Quality of Treatment 


Another issue in examining the fidelity assessment of the ITM for FFT and FFP services 
concerns treatment quality, including how well JRP is implementing these interventions, 
whether or not the appropriate dosages are provided, the adequacy of the training staff 
receives, and the extent to which treatment services vary by location within the JRP 
program and between JRP and community programs and facilities. 


Page 3 of 13 


Quality of FFT 


As a whole, the FFT therapists appear to be performing at a uniformly exemplary level, 
given the restricted resources available to them. Without exception, all of the FFT 
therapists participating in the site visits associated with this report demonstrated the high 
levels of knowledge and skills required for effective FFT implementation. Treatment 
dosages for youth and families who receive FFT appear quite good, with number of 
therapy sessions and rates of treatment retention and completion for FFT providing solid 
indicators of FFT quality. Engagement in FFT is a shared responsibility across reentry 
and other program staff, FFP counselors, and FFT therapists. Engagement appears to 
vary widely across JRP sites. Some program managers and staff reported communicating 
to youth and families that FFT is an optional service and expend little effort to engage 
youth and families in treatment. Other program managers and staff reported that 
participation in FFT is actively encouraged and pursued vigorously using FFT 
engagement and motivation strategies to achieve high levels of family engagement in 
FFT. The lack of cross-site consistency in efforts to engage families in FFT represents a 
significant weakness for the FFT implementation program. 


The overall quality of FFT within JRP ranges from moderate to high, but is diminished to 
varying degrees by different levels of administrative support provided across sites, the 
amount of FFT supervision/consultation therapists receive, and the amount of time 
therapists are required to allocate their efforts to other areas of responsibility. Across JRP 
sites, the FFT caseloads of five families is low and therapists’ skills and specialized 
training are not being applied optimally to help youth during the reentry process as 
therapists spend significant amounts of time traveling to meet with youth and families in 
distant locations. Moreover, the structure of placing FFT therapists in the role of 
supervising FFP counselors is detrimental to both FFT and FFP implementation quality 
because FFT therapists are responsible for overseeing FFP fidelity for counselors who 
do not directly report to them and who are formally supervised by others who conduct 
their performance evaluations. This situation gives FFP counselors the option to follow or 
reject the guidance of FFT therapists, limiting the impact of therapists’ time and skills. 


FFT quality is also diminished somewhat by the lack of systematic training for all 
therapists in all elements of the ITM. Because therapists are not systematically included 
in trainings for Dialectical Behavior Therapy (DBT) and Aggression Replacement Training 
(ART), two interventions included within the ITM intended for delivery to all youth during 
their period of incarceration, therapists do not have full knowledge of the clinical tools 
youth have been exposed to while incarcerated and cannot optimally guide youth to fully 
develop and implement these tools as situations arise once they transition to their 
community environments. 


Finally, traveling to distal sites to conduct FFT sessions with families is burdensome for 
the system and diminishes the time available to provide services to youth in need of 
treatment. The limitations on numbers of youth who receive FFT lowers the overall quality 
of FFT implementation. 
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Quality of FFP 


The treatment quality of FFP appears more variable, relative to FFT quality, ranging from 
low for FFP counselors with seemingly limited commitment and fidelity to the FFP 
approach and moderate for counselors who seem to have embraced the FFP model and 
appear motivated to implement FFP with integrity. Hampering treatment quality for FFP 
is the inadequacy of training, beyond the initial FFP training, the supervision structure, 
and fidelity monitoring procedures. 


Basic training in the FFP model appears to be excellent. Collectively, the trainers are 
experienced in FFT, FFP, other evidence-based interventions, working with juvenile 
justice involved youth and their families, and adolescent behavioral health. Although FFP 
counselors appear well grounded in the FFP approach, they are not exposed 
systematically to FFT training, limiting their ability to understand the work the FFT 
therapist is engaged in with youth and families. This limitation compromises the ability of 
FFP counselors to use that knowledge to collaborate with FFT therapists on goals for 
youth and families to help consolidate and apply the skills they are learning as they 
navigate the reentry period. As with FFT therapists, FFP counselors also are not provided 
with systematic exposure to DBT and ART training. Without a broader exposure, 
counselors are not fully able to understand the clinical tools youth have been taught and 
guide youth to further develop and implement these tools as situations arise after they 
transition to their community environments. Additional training in specific areas of 
substance abuse treatment and trauma coping strategies, problems affecting a significant 
proportion of youth in JRP, is also lacking. 


Treatment Implementation Variations Across JRP Locations 


With respect to variations in treatment services across JRP sites, dramatic differences 
were readily apparent for FFT implementation across sites. Relatively fewer cross-site 
differences observed for FFP. There were also some differences noted as to how FFT 
and FFP link youth and families with community programs and facilities. However, these 
differences appear due to the variations in resources available across different 
communities. For example, the urban and rural settings have different resources for 
housing, food, and other needs. In more remote areas, matching youth to vocational 
development opportunities may also be sparse, compared to opportunities in larger 
communities and urban areas. These variations appear unrelated to FFT and FFP 
implementation across sites. 


Integrating FFT and FFP with Assessments, Other Services, and Reentry Planning 


The integration of FFT and FFP with assessment and reentry planning appears very 
strong. The assessment information is an important factor, though not the sole 
determinant, for deciding which youth and families are deemed most appropriate for and 
in need of FFT services. Similarly, the assessments provide a unique source of valuable 
information to guide FFP. 
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Overall, the integration of FFT and FFP with other interventions and services within JRP 
varies widely, depending on the thoroughness and frequency of case staffing meetings 
with the entire services staff in attendance and the effectiveness of communication across 
staff within each site. Some sites have a rigorous process for integrating FFT, FFP, and 
other interventions and services for youth through their weekly staff meetings and regular 
review of all cases. These programs are exemplary. Other programs do not follow a 
process that ensures a regular review of all cases and a climate that fosters regular 
communications between staff to ensure effective coordination of services. 


The integration of FFT and FFP with other interventions and services external to JRP is 
challenging, especially with the current resources available. The wrap-around services 
program provides a model for the integration of FFT and FFP with other interventions and 
services, but is labor intensive, costly, and available for only youth at the highest levels 
of risk for recidivism and recurring problems. Thus, the current level of integration of FFT 
and FFP with other treatments outside JRP appears appropriate. 


Adequacy of Quality Assurance Plan 


Given that high competent adherence of clinicians to evidence-based treatment models 
has been empirically linked to better youth and family outcomes, while poorer adherence 
has been associated with poorer outcomes and higher recidivism rates, the adequacy of 
the quality assurance plans for FFT and FFP are crucial for maintaining clinician and 
counselor adherence and maximizing the effectiveness of the ITM. Both FFT and FFP 
have basic quality assurance plans in place and all sites appear to be highly compliant 
with quality assurance procedures. Thus, quality assurance measures are completed and 
this provides useful information about staff and program activities. The adequacy of the 
quality assurance plans, however, are limited by the general lack of specificity with 
respect to how sub-optimal FFT or FFP adherence is addressed. Moreover, the validity 
of the adherence measure, the Global Rating Measure (GRM), for measuring FFP 
adherence is quite weak. 


For FFT, the quality assurance procedures measure the fidelity of implementation of the 
treatment model moderately well. The initial training and subsequent experience of 
therapists, combined with clinical tracking system provided by FFT training program 
provides a minimum level of quality assurance. If adherence declines, however, it is 
unclear what steps are taken to guide therapists and help them return to acceptable 
adherence levels. The process for delivery of feedback to therapists is unclear and there 
are no observation-based quality assurance procedures in place by which a supervisor, 
trainer, or consultant would provide guidance to help therapists regain adequate fidelity. 
Supervision and consultation opportunities for FFT therapists appear minimal. 


For FFP, the limitation of the GRM is that, as indicated by the title, ratings cover broad 
swaths of counselor activities, providing a general and likely imprecise approximation of 
counselor behavior and performance over all sessions conducted within a 90-day window 
or, initially, within a 30-day window. The GRM is completed by the FFT therapist who 
participates in FFP implementation at each site. Because the GRM involves an overall 
set of ratings of the FFP counselor’s fidelity across multiple youth/families and across all 
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FFP sessions observed or co-led by the FFT therapist in the prior 30 or 90 days, the 
ratings essentially become a measure of relatively stable counselor “traits” rather than an 
assessment of the degree to which FFP is being implemented with fidelity in any given 
meeting the counselor has with youth and families. Because the global rating approach 
tends to aggregate specific observations of FFT behavior into a broad picture of 
performance, the adherence measure does not capture instances of specific areas of 
strength or areas in need of improvement. Moreover, because the FFT therapists do not 
have a direct supervisory role in relation to FFP counselors, their ability to provide 
corrective feedback is hampered and their influence on performance is weakened unless 
a very strong bond between the therapist and counselor have been forged. Such bonds 
were present within some sites visited and absent within others. Given the observation 
that commitment to the FFP model was somewhat variable across program staff, the need 
for strengthening the role of the FFP supervisory position is clear. 


Variations in Treatment Access by Race, Ethnicity, Age, and Gender 


Throughout the fidelity assessment of the ITM, there was no evidence of any kind that 
FFT or FFP was implemented with systematic variability across youth race, ethnicity, age, 
or gender. The site visits included attending FFP field meetings, meetings that included 
a diverse mix of youth with respect to race, ethnicity, age, and gender. FFP was 
implemented consistently and adherently across all youth and families observed. All 
interviews and interactions with staff across the three JRP sites visited and all materials 
reviewed are consistent with the conclusion that there is no variation in FFT or FFP 
implementation by race, ethnicity, age, or gender. 


One caveat to this conclusion is that it is possible that racial/ethnic minority youth may by 
overrepresented among families with fewer socio-economic resources who, due to higher 
housing costs closer to JRP sites in city or town centers, may reside in more distal areas 
within each region. Such a situation would make it more challenging to provide services 
to youth in outlying areas with the same frequency as youth residing closer to the JRP 
facilities. JRP staff, program managers, and administrators uniformly recognized the need 
to provide services to youth at the same level, regardless of their proximity to program 
facilities and expressed the commitment to ensure comparable quality and frequency of 
services for those youth living farther away. A significant problem that contributes to the 
increased potential for minority youth to receive fewer needed services, however, is the 
limit the State of Washington places on FFT therapists and FFP counselors for mileage 
reimbursement and policy restrictions that limit travel. 


Recommendations for Improving Implementation 


The recommendations for improving the implementation of FFT and FFP within the ITM 
are based on the issues noted in previous sections. Improvements in implementation in 
many areas are feasible with the current level of resources provided to parole aftercare. 
A primary concern is that only half of the youth are placed in JRP upon their release from 
incarceration. JRP could have a far greater impact if each region maintained oversight 
and supervision of a// youth re-entering their communities. Within such an approach, the 
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assessment process could guide the allocation of FFT, the amount of contact youth and 
families have with FFP counselors, and other JRP services. 


Although this would require a systemic change at the legislative level, such a change 
could be initiated at a minimal level without an increase in funding. Some JRP sites 
already strive to serve youth not formally placed within their oversight, juggling resources 
internally to cover needed services. In addition, costs associated with assessment, 
placement, and reentry planning for two groups of youth could be re-distributed to JRP 
programs to allocate internally. As noted above, monitoring and providing services to non- 
placed youth would also likely result in substantial savings to the state through decreased 
recidivism and re-incarceration. Although attempting such a change without an increase 
in funding would not produce optimal results, the empowering JRP to allocate resources 
across the single continuum of need would help guide the application of current resources 
more efficiently. 


Supervising all youth transitioning to the community would double the overall number of 
youth served, but would not require a doubling of the workload for JRP staff. A substantial 
number of lower risk youth would need less monitoring and could be supervised with 
monthly FFP check-ins and brief phone contacts. Similarly, youth with moderate risk, 
including those who would have been placed in JRP and those who would have been 
released without JRP placement, could be supervised with bi-weekly FFT check-ins, with 
phone contacts as needed, while higher risk youth could receive services as they are now 
provided by JRP. Allowing adjustments in service allocation to made internally by JRP 
would significantly enhance the quantity and quality of services across all youth. Because 
all youth not placed in JRP are released from incarceration with some arrangement for 
housing, funding for housing services would not necessarily need to increase. 


Another issue that could be improved is cross-site consistency of FFT and FFP 
implementation. The development of checklists, procedures, and/or benchmarks to 
monitor cross-site consistency could help to ensure that comparable levels of family 
engagement in FFT are achieved across sites, the frequency of reviews of cases by all 
staff are consistent across sites, and comparable services and treatment dosages are 
delivered to equivalent proportions of youth and families across sites. Possibly, 
supplemental training and consultation may be needed to assist sites with lower FFT 
engagement rates to improve FFT implementation. In addition, changing the supervision 
structure so that FFT therapists who guide FFP counselors also have a meaningful role 
in supervising counselors, contributing to their performance reviews, and providing input 
into merit increases would provide much needed support for maintaining more cohesive 
teams of counselors and enhance FFP fidelity. 


A more rigorous fidelity monitoring measure is needed to improve the quality of FFP. 
Rather than using the GRM as a gross measure of FFP adherence across sessions, 
families, and longer periods of time, the GRM could be adapted as a brief tool that 
requires only a few minutes for counselors to complete at the end of each session. On 
co-visits, the FFT therapists would also complete the brief tool and a supervision could 
include a review of counselor and supervisor adherence forms as a focus for feedback 
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and coaching. Adherence tools have been developed for FFT using mobile app 
technology to save therapist time, maximize efficiency, and lower costs for sustaining 
fidelity. Such tools could easily be adapted for an FFP adherence measure, yielding an 
approach more similar to the adherence procedures completed by FFT therapists. 


Implementation of FFT and FFP could also be dramatically improved at relatively low cost 
(i.e., release time for therapists and counselors) allowing all therapists and counselors to 
attend or “audit” ongoing trainings for DBT and ART. Such a step would ensure FFT 
therapists and FFP counselors all have a basic knowledge of a broader set of 
interventions provided within the ITM intended for all incarcerated youth. FFT quality 
could be improved by expanding therapists’ knowledge of the clinical tools youth have 
been exposed to while incarcerated to better guide youth to fully develop and implement 
these tools as situations arise once they transition to their community environments. 


Additional Resources to Improve Treatment Quality and Integration 


Some improvements for FFT and FFP will require additional funding. One readily 
apparent area for improvement is the need for additional personnel. The number of FFT 
therapists and FFP counselors is inadequate. The quality assurance of FFT could be 
enhanced significantly by creating teams of two or more therapists within each JRP site, 
providing regular supervision (e.g., weekly) and consultation (e.g., monthly) for these 
therapists, and providing expanded opportunities for observation-based coaching and 
feedback. This would improve FFT quality, within-site cohesion, and expand treatment to 
more families. With more nimble therapists with advanced training in key areas of 
behavioral health (i.e., - see training recommendations below) auxiliary support staff or 
community professionals funded by the state to service this population and currently 
providing substance abuse or mental health treatment could be reduced to allow for the 
addition of FFT therapists. Additional FFP counselors are also needed at all sites to 
improve the frequency and duration of FFP sessions with youth and families. By 
expanding the FFT and FFP staffing levels, overall program cohesion would be increased 
as the number of specialty staff decreases and the FFT and FFP teams provide a 
coordinated and integrated set of services. 


Another clear need is funding to increase FFT and FFP staff salaries. FFT therapists in 
JRP are paid less than FFT therapists in other branches and divisions of the state 
government. Lower pay means the best staff leave to take higher paying positions in other 
departments and the disparate pay contributes to lower morale. Parole counselors in JRP 
are among the lowest paid in the adult or juvenile justice system. Comparable, 
competitive pay for FFP counselors is essential to attract and retain the top staff and 
sustain effective evidence-based programs. 


The travel restrictions and mileage reimbursement limitations specified by state policies 
hamper the delivery of FFT services. When FFT travel limits have been reached, 
therapists are compelled to find creative ways (e.g., “borrowing” unused miles from staff 
with lower travel requirements) to meet the needs of youth and families. One solution 
would be to provide additional resources to FFT therapists for travel. This solution does 
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not address the issue that time spent traveling restricts the number of youth FFT 
therapists can serve. Because of the long distances that must be traveled to meet with 
youth and families in all regions throughout the state, tele-health capability is needed to 
expand the reach of FFT and FFP to more youth and families and increase the frequency 
of contacts and total treatment dosage for youth. Tele-health programs have been used 
effectively throughout the country and would allow for many more FFT and FFP sessions 
to be conducted via secure video conference than is currently possible. Tele-health 
capability would allow for considerable increases in FFT and FFP caseloads and allow 
staff skills and specialized training to be applied more efficiently to help youth during the 
reentry process. 


Training FFT therapists and FFP counselors to integrate strategies from evidence-based 
approaches, particularly interventions for substance abuse treatment, depression, and 
trauma-focused interventions representing the greatest areas of unmet need among 
youth in JRP, would dramatically improve the treatment quality and expand the impact of 
FFT and FFP. In addition, training auxiliary program staff in FFP would strengthen the 
overall cohesion within each JRP site. Additional training for FFT and FFP staff would 
reduce the need for WISE services and specialized staff to provide substance abuse or 
other treatment services, resulting in cost savings. The FFT and FFP models are 
designed for flexibility and can be tailored to meet a wide range of needs of youth and 
families. FFT has systematically been implemented and evaluated as a substance abuse 
treatment (Waldron et al., 2001; Waldron & Turner, 2008; Waldron, Brody, & Hops, 2017) 
and the strategies for treating youth who are substance-involved could readily be 
integrated into FFT throughout JRP, eliminated the need for independent substance 
abuse counseling staff and making more efficient use of resources. Similarly, FFT 
strategies for depression and trauma-related problems are well developed and 
specialized training for FFT therapists would help diminish the need for more intensive 
services, including individual psychotherapy and some wrap-around services. 


Another essential improvement would be to add capacity within each JRP site to conduct 
data analyses to monitor program performance (treatment quality, quality assurance, FFT 
and FFP outcomes) internally. Having analytic tools and staff to use them would allow 
sites to make adjustments and improvements more flexibly and nimbly than is possible 
when relying on independent studies to be conducted and fidelity assessments to be 
completed when the data become less useful. Having technology in place to support 
electronic entry of adherence ratings would provide immediate access to reports, 
facilitating analysis needs at low cost and supporting internal tracking and program 
sustainability. 


FFT and FFP for Emerging Adults Served by JRP 


The period of emerging adulthood for youth between the ages of 18 and 25 is one of rapid 
and extensive transitioning as youth experience physiological, sexual, cognitive, and 
emotional changes. Emerging adults often take on new adult roles and responsibilities, 
including advanced education or occupational training, entry into the workforce, exiting 
their family home, forming new intimate relationships, and starting their own families. 
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Many young adults also move into new adult roles and responsibilities. They may begin 
higher education studies, enter the workforce, move away from home, or start a family. 
They will experience increased opportunities to make their own decisions, including legal 
decisions, and take on increasing or exclusive financial responsibility for themselves. 


This period is often overwhelming, but is expected to be particularly difficult for youth 
challenged by reentry into their communities. It is not surprising that currently more than 
a third of youth released from incarceration to live with their families of origin are 
reportedly no longer living with those families six months later. A significantly greater 
percentage of reentering youth aged 18-25 are likely to live away from their families and 
have less social support from their families unless concerted efforts are made through 
FFT and FFP to strengthen familial connections. JRP resources will undoubtedly be 
strained heavily by the expanding of services up to youth age 25, particularly because 
these youth will not only continue to need all of the services currently provided within JRP, 
but these youth will have fewer safety nets for the basic needs of food and shelter. JRP 
should anticipate escalating challenges to facilitate housing, transportation, community 
services (e.g., food banks), and resources for employment and occupational development 
for these older youth. The family is a key source of support to alleviate the burden that 
JRP will experience, thus ensuring high quality FFP for all youth and expanding FFT toa 
greater portion of youth in need of treatment services will empower families as partners 
to meet the challenge. 


Qualifications 


The author of this report, Holly Barrett Waldron, Ph.D., has an extensive background in 
FFT and prior experience with FFP that qualify her to conduct a fidelity assessment of the 
FFT and FFP components of the ITM. Spanning four decades of work, she has been an 
FFT innovator and model developer, training and supervising hundreds of FFT therapists 
in the United States and in Latin America. She is proficient with the FFP approach through 
her prior work with the FFP development team, experience designing clinical trials to 
evaluate the effectiveness of FFP, and consultations with juvenile services departments 
considering the implementation of FFP. Currently, Dr. Waldron is a Senior Scientist and 
the Director of the Center for Family and Adolescent Research at the Oregon Research 
Institute. She is also the President and CEO of LIFFT, Co., an organization focusing on 
technology transport and implementation services, expanding the reach of evidence- 
based treatments and training programs for youth with co-occurring behavioral health 
problems. Dr. Waldron is a licensed psychologist (New Mexico License 1825) and has 
more than 35 years of clinical, research, and treatment dissemination experience. She 
has worked extensively with children, youth, and families to address behavioral health 
problems using evidence-based treatments. She has also collaborated with the Juvenile 
Justice System, schools, and other community programs for youth to help troubled youth. 
As a researcher, she has led many treatment development programs and randomized 
Clinical trials examining family-centered and cognitive behavioral therapies for adolescent 
substance use disorders, depression, disruptive behaviors, trauma-related problems, and 
HIV risk behaviors. Her treatment research efforts have also focused on evaluating 
parenting approaches, contingency management, motivational interviewing, juvenile 
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justice interventions, and web-based treatments. As a scientist-practitioner, Dr. Waldron 
is actively engaged in implementation science research and treatment dissemination 
activities. Her current dissemination and research activities focus on the use of 
technology to improve the efficiency and lower the costs for FFT training, supervision, 
and fidelity monitoring. She has also developed an FFT tele-health service delivery 
approach to enhance the reach and sustainability of FFT and other evidence-based 
treatments in community-based treatment settings nationally and internationally. 
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About Aggression Replacement Training (ART 

Youth violence has a significant impact on communities in America. There are continuing 
efforts to identify treatment programs to reduce youth violence. One such program is 
Aggression Replacement Training (ART). The program was originally designed for hostile and 
assaultive youth who were incarcerated. The program is a 10-week psycho-educational 
intervention. A series of prosocial behaviors are taught through a structured leaning 
environment to small groups of 6 to 8 youth. The training consists of three components, social 
skills, anger control, and moral reasoning. In 1987, Glick and Goldstein first demonstrated that 
ART reduced the number and intensity of behavioral incidents of youth while in the institution 
and showed that youth were able to apply and transfer these skills after transferring into the 
community. This program showed promise and has been implemented in many different 
settings in the decades since its original implementation. Adaptations have included school 
settings, family-based, and for youth on the autism spectrum (Roth, 2003; Calame, 2003; 
Moynahan, 2003). 

In 2016, Brannstrom and colleagues conducted a systematic review of the literature. 
They identified 16 studies that examined the impact of ART on recidivism, primarily. The review 
concluded that, “the primary studies of ART do not provide a sufficient base for substantiating 
the claim that the program is effective for reducing antisocial behavior in adolescents and 
adults” (pg. 40). Among the major concerns was that almost halfF of the studies were 
completed by researchers who had a vested interest in the program. Additionally, the 
methodological rigor of the current body of research is limited. CrimeSolutions.gov rates ART as 
an effective program? based on two studies (Barnoski, 2004; Gundersen and Svartdal, 2006), 
however, more recent research is not conclusive about the effectiveness of the program. 

In Washington State, there has been significant investment in ART. The program was 
first implemented for justice-involved youth in 1999, in a community setting. It was later 
expanded and implemented within the state juvenile residential facilities in 2008. There have 
been a series of studies on Washington State ART (WSART), with the most recent studies 
causing concern about the effectiveness of the intervention. In 2004, Barnoski evaluated 
WSART by comparing youth who started the program in 2000 to those who were eligible, but 
were on the waiting list due to a lack of available resources. The study found that ART resulted 
in a marginally significant reduction in 18-month recidivism; however, for those courts that 
were determined to have administered WSART competently, there was a significant reduction 
in felony recidivism. This finding highlighted the importance of implementation fidelity. 

In 2017, using propensity score matching, Peterson (2017) compared youth who started 
WSART to those who did not receive an evidence-based program. While the author notes some 
data limitations, the study found that youth who started ART had higher felony recidivism than 
a matched control group. Most recently, in 2019, Knoth, Wanner, and He, using a sample from 
2006 to 2016, found that WSART participants were significantly more likely to recidivate than 
matched youth who did not participate in the intervention. The authors compared WSART to a 
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treatment as usual group using propensity score matching. Further, the authors found that the 
trainer competence score did not have an effect on the results. 

All the findings on WSART come from the implementation through the courts. Little is 
known, however, about the effectiveness of the program for juveniles in residential programs. 
The training, quality assurance protocols, and assessments in the Juvenile Rehabilitation 
institutions are identical to those that have been used in the courts, with youth who are in the 
community. The current assessment explores the implementation of ART in the three JR 
institutions (Echo Glen Children’s Center, Green Hill School, and Naselle Youth Camp). From 
January 2008 to December 2019, JR had over 3,200 starters in ART in one of the institutions. For 
this assessment, all manuals, quality assurance plans, training materials, and surveys were 
reviewed. Additionally, the ART administrator, JR master trainers, and national experts were 
interviewed for their insight on the current implementation of the program in JR specifically, 
and the model more generally. 


Assessment questions: 


The current assessment seeks to address the questions listed below. These are common 
questions that were asked about all the treatment areas in the Integrated Treatment Model as 
part of the current fidelity assessment. 


1. Are youth being appropriately matched with treatment based on need? 
a. What assessment is being used for program eligibility? 
b. Is the assessment being used appropriately? 


In Juvenile Rehabilitation the Integrated Treatment Assessment (ITA) is used to 
determine eligibility for Aggression Replacement Training (ART). The ITA is a risk and needs 
assessment that is administered within 14 days after admission into a JR residential facility. The 
assessment is similar to the Residential Positive Achievement Change Tool (R-PACT), which has 
been studied extensively. However, JR has yet to validate the weighting of the ITA for the JR 
population. Nevertheless, ART in JR uses the eligibility criteria identified by WSART (which is 
implemented in the courts, with youth living in the community). Specifically, to be eligible for 
ART a youth must have a high score in domains: 1 (record of referrals), 10B (current attitudes 
and beliefs), 11 (aggression), and 12 (skills). 

It is encouraging that JR is using the ITA to determine eligibility for ART. This follows the 
risk-needs-responsivity model (RNR). The RNR model is comprised of three principles. First the 
risk principle, which suggests that those with the highest risk for reoffending should be 
prioritized for treatments and other interventions. Second, the need principle recommends that 
the individual needs of each youth are determined, specifically those needs are most likely to 
be associated with criminal behavior. And third, the responsivity principle requires that the 
correct type of programming be offered based on an individual’s risk and need profile (Crites 
and Taxman, 2013; Brogan, Haney-Caron, NeMoyer, and DeMatteo, 2015). In ART in JR, the use 
of the ITA indicates that there is some attention being paid to the need and responsivity 
principles of the RNR model. 

None of the research reviewed indicates that the current eligibility criteria is predictive 
of future violent behavior, which would require ART as an appropriate treatment. The current 


program is using an assessment for eligibility, and the assessment is being used appropriately, 
however, JR has not determined that the eligibility criteria that have been selected are 
appropriate. The research has not determined who will benefit the most from the program. JR 
needs to inform staff of when ART is the best response, given a profile of risks and needs of a 
youth. If the program has the wrong eligibility criteria, an outcome evaluation might determine 
the program is ineffective, when the reality is that the program was given to the incorrect 
youth. The program needs to answer, which youth need ART and which youth can ART impact 
the most? The answer to these questions will help JR refine their eligibility criteria, and ensure 
that resources are used most efficiently. 


2. Is the treatment high quality? 
a. How well is JR implementing this treatment? 
b. Are youth receiving the appropriate dosage? 
c. Do staff receive the correct type and amount of training? 
d. Does there seem to be variation by location? (between institutions, between 
institutions and community facilities) 


JR appears to be implementing the treatment according to the design. There are strong 
training and quality assurance protocols in place. In terms of dosage, all those who are start the 
program receive the same dosage, which is three sessions per week for 10 weeks. It is not clear 
whether youth are receiving the right dosage. It is likely that some youth require more 
treatment and some less, however, the current design of ART does not allows for this type of 
dosage variation. JR should consider a graduated or extended course of ART for those youth 
with greater need and who are at a higher risk for future aggression. There was some reporting 
that ART is only allowed 45 minutes for sessions in some places due to school schedules. This 
would result in a lower dosage than intended. There is variation by location in terms of when 
ART is administered but the standards and quality assurance for the program is consistent 
across location. 


3. How well is the treatment integrated with the assessments, other treatments, and 
reentry planning? 


The current implementation design does not include much integration of the 
assessment or integration with other treatments or reentry planning. Assessments are used to 
determine eligibility, however, they are not used during ART to determine dosage or focus, 
since it is a group based program. ART has a unique set of skills that are different from 
Dialectical Behavior Therapy (DBT), and there is little to no training for staff on how the two 
sets of skills overlap or are complementary. Generally, living unit staff have not been trained in 
the ART skills and are not be able to reinforce youth who use the ART skills. 


4. Does the current quality assurance plan adequately measure the treatment model? 


WSART has a well-established quality assurance plan. Facilitators submit a recording of a 
session annually and this session is evaluated by a master trainer. As a result, the facilitator is 


determined to be adherent to the model or not. There is a youth survey in place, however, this 
is not be administered on a routine basis. The program also has an established curriculum for 
training new trainers, monthly consultation conference calls, site visits and feedback from the 
ART administrator, and semi-annual quality assurance committee meetings. 

While there are well-established quality assurance protocol currently in place, there is 
not a process to determine whether the program aspects being monitored are related to 
improved outcomes. ART in JR needs to establish which implementation metrics are most 
associated with improved outcomes so that the treatment administrator can monitor those 
metrics on a regular basis. 


5. Do you see any evidence that would concern you that treatment access varies by race, 
ethnicity, age, or gender? 


| do not see any evidence that treatment access varies by race, ethnicity, age, or gender. 
Youth in JR have had pretty broad access to ART. 


6. What recommendations do you have to improve implementation given the current level 
of resources? 


There are a number of recommendations that could be started immediately, with little 
to no additional resources. First, the eligibility criteria should be better defined. Currently, any 
youth with a score above zero in any of the specified domains is eligible for ART. Low scores in 
these domains indicate lower risk. JR should set cut off values for a more clear eligibility criteria, 
that will allow staff to prioritize those youth with the greatest risk and the most need in this 
area. Clearly, research will need to inform these criteria in the future, but in the short term, 
these can be made more clear. 

The number of youth who get ART should be based on need and not predetermined 
quotas. This can be a challenge when trying to plan for treatment capacity, but ultimately need 
should drive capacity planning. More treatment is not always better; instead, the correct 
treatment is best. JR should refine the eligibility criteria so that it is based on evidence 
accumulated over the past 10 years of implementation, and then focus ART on those with the 
most need and the highest risk. 

At some of the institutions, because of the school schedule, only about 45 minutes is 
permitted for the ART sessions. Staff should reconsider how ART is being implemented to 
ensure a full hour for programming for ART sessions, to ensure implementation fidelity. 


7. What additional resources would you recommend to improve treatment quality and 
integration? 


The first priority is to better understand how well ART is working in JR. Given recent 
findings that suggest an iatrogenic effect of ART when administered in the community with 
justice involved youth, JR should first ensure that this is not the case with those receiving the 
treatment in residential facilities. Next, research should investigate the optimal eligibility 


criteria. Programming will be the most effective when matched to the youth with the risk and 
need profile most appropriate for the treatment. 

JR should set up a curriculum review committee for ART, but this could be for the ITM 
more generally. This committee would be responsible for reviewing treatment curricula and 
ensuring that they are up to date and relevant. Many of the lessons in ART have been in place 
for decades, and experts need to have a process to review their content to make sure they are 
culturally relevant for the target population. Lessons can be reviewed on a rotating basis, so 
that over the course of a few years, the entire curriculum is reviewed and updated. 

JR needs to implement a process that will allow for closely monitored variations of ART 
programming. For example, JR could test a shortened version of ART for youth who have the 
need and are high risk, but do not have a very long sentence. JR could also start to explore 
options for a graduated course for youth with longer sentences, who need additional 
reinforcement of the ART skills. This might include an alumni group that continues to meet so 
that the skills can be reinforced. There is also some discussion of whether an 8 week course 
could be just as effective for youth. These are all important considerations, and JR needs to 
have a process to continue to learn about what is most effective for youth who are high risk 
and have the need to reduce aggression. These types of variations would need robust 
evaluation protocols to help determine both the strengths and challenges of each. 


8. JR will be serving youth up to age 25 in the future. What advice would you give the 
agency as they prepare to provide this treatment for older youth? 


The first consideration is the assessment used for ART eligibility. The ITA was developed 
for juveniles and validation work has not been done for an adult population. JR needs to 
determine whether the risk and need profile of a juvenile is the same as the profile for an adult, 
in terms of whether ART is the appropriate program. Also, some stakeholders have indicated 
that ART has been offered in the Department of Corrections. As youth are brought back to JR 
from DOC, case managers need to identify if an individual was already taken ART in DOC. 
Individuals should be re-assessed to determine the treatment that will address their current 
needs. 
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Introduction 


This brief report summarizes findings from an assessment of programming delivered to children and 
adolescents in the care of the Washington Department of Children, Youth, and Families (DCYF). 
Specifically, this assessment reviewed policies and practices specific to youth who sexually offend. It 
concludes that the DCYF has developed a very good platform of care that is in line with best practices 
and the most recent research. This report offers suggestions for making this programming better still. 
This report summarizes what the author stated to DCYF administrators on December 2 and 3, 2019. 


By way of context, | (the author) have produced numerous books, book chapters, and articles in the area 
of understanding, assessing, and treating sexual violence. I’ve trained and lectured in these areas around 
the world, and my work has been translated into five or more foreign languages. | am the 2014 recipient 
of the Distinguished Contribution award of the Association for the Treatment of Sexual Abusers (ATSA) 
and the 2018 recipient of the C. Henry Kempe Lifetime Achievement award from the National 
Adolescent Perpetration Network. 


The questions that this writer was tasked with included: 


1. Are youth being appropriately matched with treatment based on a risk-needs-responsivity 
framework 
a. What assessment is being used for program eligibility? 
b. Are assessments being used appropriately? 
2. Is the treatment high quality? 
a. How well is DCYF implementing this treatment? 
b. Are youth receiving the right dosage? 
c. Do staff receive the correct type and amount of training? 
d. Does there seem to be variation by location? 
3. How well is the treatment integrated with the assessments, other treatments, and reentry 
planning? 
4. Does the current quality assurance plan adequately measure the treatment model? 
5. Is there evidence that treatment access varies by race, ethnicity, age, or gender? 
6. What recommendations are there for improving implementation given the current level of 
resources 
7. What additional resources are recommended to improve treatment quality and integration? 
8. What advice is there for serving youth up to 25 years old? What kinds of preparations can DCYF 
make? 


The Broader Picture: Program Assets and Attributes 
The DCYF program, particularly as delivered in its inpatient facilities, emphasizes a strong skills-based 


approach using empirically supported treatments such as Dialectical Behavior Therapy and Aggression 
Replacement Training. The administration retains a number of consultants to ensure that these 


treatments are provided with fidelity to the model. In a cursory review of the programs, this was in 
evidence (for example, the author observed part of a treatment group and found evidence of treatment 
lessons on the walls of programs, serving as helpful reminders to the clients). This integration of solid 
treatment approaches into the milieu speaks to the level of intensity of the services provided. 


It was obvious through interviews with program staff in several locations that the administration very 
genuinely cares about effective treatment for the youth in their care, and for the safety and wellbeing of 
those the same youth have harmed. There is no program measure for this; rather, the author is 
comparing our interactions with those in other programs inside and outside the US in similar assessment 
processes. Program administration, almost by definition, means balancing often competing elements 
(such as ensuring quality of care within a tight budget). It can be very easy for administrators to err on 
the side of appearances rather than the substance of effective treatment programming. It was very clear 
that the current administration has the youth (and families) they serve at the top of their priorities. This 
is a crucial asset that should be recognized. 


DCYF programming involves a sophisticated electronic medical record that appears to be effective for its 
current needs. As is always the case in these matters, it is not perfect and people have occasional reason 
to complain, but on balance the program materials are in place and accessible. This is not often the case 
in other programs. 


Likewise, the “golden thread” of information between treatment plans and case notes was in place and 
effective. The author reviewed a number of treatment plans and case notes on a random and semi- 
random basis and found that none were problematic, and that the structure of each allows for 
continuity of treatment communication. In other words, case notes reflected treatment plans, which in 
turn reflected the medical and/or psychiatric evaluations from which they were drawn as well as the 
legal documents for each youth. Case notes were generally well written and the treatment plans 
themselves reflected and enabled an individualized treatment approach for each youth in treatment. 


Particularly impressive was the separation of group treatment interventions from the kind of deeper 
discussions about offending that take place in private. There is a wealth of research showing that group 
treatment with youth is most effective when it is highly structured, and that in-depth discussions about 
topics such as family relationships and offense disclosure are best left to individual counseling sessions. 
Again, this is not always the case in programs. 


Further, the program is currently working to refine its programming in accordance with developments 
elsewhere in the field. In particular, the program is working to improve its capacities in trauma-informed 
care and the use of treatment goals that clients in treatment can “approach” rather than “avoid”. Very 
simply put, a body of research finds that people are motivated by goals that they can acquire and 
achieve rather than those they must avoid. “Having a balanced, self-determined lifestyle where | can get 
what | want without hurting people” is far more desirable to people who have abused than “Living by a 
set of rules that will make it impossible for me to re-offend.” 


Finally, although the author was able to interview two clients directly (one randomly), it was clear 
through observation that the youth | observed were engaged within the program. Likewise, the staff 


with whom | interacted (some by chance, others by design) were committed to their work. 


On balance, there is much to be proud of within this branch of DCYF services. 


Challenges 


The number one concern expressed by all interviewed is that the treatment provided within the 
inpatient components of the program is not provided by licensed clinicians. Although the programs are 
assisted by psychologists, the treatment itself is delivered by people who do not have specialized 
training in psychotherapy. This point needs to be understood in the proper context: the programs 
themselves have accommodated this fact to the best of their abilities. As mentioned above, the 
inpatient-based treatment is highly structured and skills-based, with the deeper psychotherapeutic 
practices taking place after the clients return to the community. Many of the primary approaches by 
which treatment is provided, DBT and ART, can be administered by professionals who are not licensed 
to provide psychotherapy under the right conditions. However, there is little question that the inpatient 
programs could become more effective with the inclusion of licensed mental health practitioners. This is 
in line with the most recent meta-analytic research by Theresa Gannon and her colleagues on the 
effective treatment of adults convicted of sex crimes. In that study, the presence of a licensed 
psychologist providing treatment was one of the defining factors of effective treatment programs. In the 
author’s opinion, finding clinicians is likely the greatest need for the DCYF programs in the long term. As 
a part of these efforts, it should always be kept in mind that this is highly specialized work where public 
safety and the health and welfare of young people hang in the balance. 


The second most cited concern is staff turnover. This is an issue that bedevils programs around the 
world. The wages are rarely competitive, while the demands are stringent, the clients caustic, and 
accountability is high at all levels. Despite children and adolescents being a particularly vulnerable 
population and the potential threat to public safety of uncared-for youth returning to the community, 
this remains a challenge for DCYF and other programs. 


The third most cited challenge is the upcoming influx of clients between the ages of 18 and 25 due to 
changes in policy. The author suspects that the logistical aspects of this change will be more difficult 
than adapting the clinical program, but it is understood that this will be a challenge. 


A large-scale challenge for the programs is that they are very often in a position where providing 
adequate treatment is simply not an option. The author heard many instances in which clients were in 
placement for only a brief period of time. Under these conditions, it can be difficult to chart a 
meaningful way forward, since some forms of treatment can make matters worse when terminated 
early. The DCYF program for youth who sexually abuse will want to develop alternative programming in 
these situations where time is limited. One option may be to develop a comprehensive assessment for 
future providers and to include the youth as much as possible in the process so that he can, in essence, 
help to assess his own life and determine the difference between where he is and where he wants to be. 


Finally, although the programs that the author were generally clean, there was some degree of clutter in 
offices and on desks (piles of paperwork, boxes for board games, etc.). The programs would benefit 
from some “sprucing up”. 


Areas for Consideration 


To this point in the review, it is clear that the basic materials of the DCYF program generally comport 
with current best practices, and that the challenges faced are not, on balance, unusual compared with 
other programs. What follows are recommendations for strengthening the clinical components of the 
program. 


First, the program’s push towards trauma informed care (TIC) will be welcome, and | suggest giving it 
top priority. To that end, a book chapter on the topic is attached to this report in case it is helpful. As the 
administration knows, TIC is a paradigm for treatment that recognizes the widespread effects of trauma 
and seeks to prevent re-traumatization. This last idea is important, as even the best designed programs 
can be re-traumatizing under the wrong circumstances. It is often difficult for outsiders to understand 
the effects of institutions and institutional life on young people. 


Next, the programs will wish to increase the amount of contact that clients have with their families 
while in inpatient care. This can include family-oriented interventions or simply increasing contact. 
Factors that appear to hinder progress in this area include that clients and their families often appear 
alienated from one another and don’t necessarily press for closer contact. This can lead program staff to 
the faulty assumption that family contact needn’t be a priority. Actively taking the lead in including 
families (often the province of clinical staff), and maintaining their involvement, could go a long way 
towards preparation for community re-entry as well as contributing to healthier functioning for families. 
Although every family is different, one option that could be considered is the use of videoconferencing 
through HIPAA-compliant platforms such as Zoom. 


Further, the staff of the programs would benefit from ongoing training in collaborative, person-centered 
approaches such as motivational interviewing. Although the staff receive training in related areas such 
as de-escalation, focusing on communications for daily interactions can also improve outcomes. It is to 
DCYF’s credit that programs have a small nucleus of staff who have natural talents in this area, typically 
cultivated over years of practice. Active training in this area would behoove DCYF and its clients alike. 


The DCYF programs can also consider the use of Routine Outcome Monitoring (ROM) to track outcomes 
related to client wellbeing and program effectiveness. The author recommends using an approach such 
as Feedback-Informed Treatment (FIT), which includes easy-to-use measures in tracking changes in 
wellbeing and in the therapeutic alliance. Together, these measures can offer information that can 
predict when cases are heading off track and provide evidence that treatment is working. FIT also 
provides insights for professional development. As easy as it can be to import and use these measures, 
sustaining implementation can be a challenge (as it is with any other evidence—based practice) and 
improvement of services can take time. ROM can be hard work. 


The DCYF programs will also benefit from continued adaptation to special-needs clients. This includes 
those with developmental delays and those on the autism spectrum. The author is aware that efforts in 
this area are under way. 


One area to consider is how best to address issues related to viewing adult pornography. Although the 
law prohibits youth under 18 from viewing pornography, it is nonetheless ubiquitous. Further, this will 
become even more of an issue once the influx of young men and women 18-25 arrive in programs. 
Clearly, DCYF will not allow pornography use in its programs. However, some degree of education can 
serve an important function in building healthy masculinity and responsible sexuality. One place to start 
might be in the use of the Savvy Consumer model (https://blog.atsa.com/2019/10/the-savvy-consumer- 
guide-for.html). 


Likewise, although sex education can be controversial, clinical interventions targeting sexual 
responsibility can be highly useful in preventing re-offense. One option might be these materials by 


Joann Schladale: http://resourcesforresolvingviolence.com/publications/. 


Finally, as a part of creating holistic, trauma-informed treatment, it may be useful to include adjunctive 
“embodied” treatments such as trauma-sensitive yoga and trauma-sensitive weightlifting. The programs 
already contain a mindfulness component, and so these could be helpful additions to the existing suite 
of interventions. 


Summary Judgments 
The questions that this writer was tasked with included: 


1. Are youth being appropriately matched with treatment based on a risk-needs-responsivity 
framework 
a. What assessment is being used for program eligibility? 
b. Are assessments being used appropriately? 


The short answer is that the answers are “yes”. However, as noted above, the circumstances of each 
youth’s involvement can prevent the strictest adherence to these principles. For example, if a higher-risk 
youth is only placed in the facilities for two weeks, this prevents an adequate period of stabilization and 
may compromise the risk principle. In some cases, youth may be kept in inpatient care longer than the 
treatment team might otherwise wish for and have to wait for the more intensive level of services 
provided by outpatient practitioners. Within the scope of their mandate, however, the program works 
diligently to adhere to these principles. 


The assessment methods for the inpatient treatment are appropriate for the services provided. Because 
the more thorough clinical work happens in community settings, the methods used for eligibility are 
appropriately idiographic. In other words, they are balanced assessments of each youth and their 
current behavioral needs. In the context of DCYF, they are used appropriately. 


2. Is the treatment high quality? 
a. How well is DCYF implementing this treatment? 
b. Are youth receiving the right dosage? 
c. Dostaff receive the correct type and amount of training? 
d. Does there seem to be variation by location? 


The overarching answer is “yes.” Given its resources, DCYF’s implementation efforts are admirable. Its 
primary asset in implementation is its use of consultants to ensure fidelity of treatments such as DBT 
and ART. This is something that the Clinical Director appears to take very seriously. The dosage question 
is addressed throughout this report, with the ultimate answer being, “Yes, within the context of 
parameters that the programs can control.” Due to the nature of the curricula delivered, it appears that 
the staff members do indeed get the correct type and amount of training, with suggestions offered 
above on how this might be improved. The variation of training appears commensurate to the special 
needs of each location, indicating that although there is some variability, it is in the service of adaptation 
to the unique needs of each setting. 


3. How well is the treatment integrated with the assessments, other treatments, and reentry 
planning? 


Overall, the integration of assessment, treatment, and re-entry planning, especially in the context of 
DCYF’s available resources, is quite clever and innovative. The assessments are individualized and serve 
as the basis of treatment in the programs. Further, the clinical leadership ensures continuity of care 
between the inpatient and outpatient components of the programs. A random review of documents 
found that each was well-written and displayed clear evidence that each component of treatment built 
on the others. Treatment notes clearly reflected the assessments, and re-entry planning clearly reflected 
treatment. In the author’s experience, this can be quite difficult to accomplish in settings such as these. 
This area is viewed as an accomplishment that DCYF can and should be proud of. 


4. Does the current quality assurance plan adequately measure the treatment model? 


Yes. The clinical administration has systems in place to ensure that the treatments delivered are done so 
with fidelity to the models used. At the time of this writing, the administration was looking into further 
developments in this area. 


5. Is there evidence that treatment access varies by race, ethnicity, age, or gender? 


No. In fact, the author was impressed by how members of the staff and administration talked about 
their efforts in these areas. The nearest to disparity that the programs get is that clients in treatment are 
disproportionately from minority backgrounds, while clinical staff are disproportionately white. 
However, there was no evidence that even this made a marked difference in the treatment experience 
of the youth. 


6. What recommendations are there for improving implementation given the current level of 
resources 


As mentioned earlier in this report, advancements in the areas of trauma-informed care, motivational 
enhancement, and the further development of approach goals would be welcome. 


7. What additional resources are recommended to improve treatment quality and integration? 


On-site clinicians for treatment provision in the residential programs would be ideal, although it is noted 
throughout this report that this has been very difficult to accomplish historically. 


8. What advice is there for serving youth up to 25 years old? What kinds of preparations can DCYF 
make? 


Clearly, the clients served by DCYF vary dramatically not only in their physical age, but also in their 
developmental abilities. This will doubtless also be true with the transitional age clients that are soon to 
arrive. Important to remember is that adolescents do not become adults overnight at the age of 18, and 
so services for these clients, like those who are younger, will need to be developmentally appropriate. 


Areas of consideration will include: 
e Discharge/re-entry planning will need to occur from the start of treatment, with an additional 


focus of finding community housing and employment that has historically been less of a focus 
with adolescents. Locating community resources can be a challenge. Some agencies have hired 


people who specialize in locating housing and employment for these clients. This will likely be 
the biggest challenge that DCYF faces. 

e Likewise, preparing clients for community challenges such as housing and employment will also 
be vital. One possible resource that can help to guide efforts during re-entry is the Circles of 
Support and Accountability model. 

e Insupport of the point above, all interventions that focus in the area of interpersonal 
competence will be welcome. A key goal can be helping clients to experience themselves as 
competent in a wide range of situations and relationships, able to relate to others empathically. 
This is often less of a focus with younger adolescents who are working more towards school and 
family re-entry. 

e DCYF may wish to consider policies as to what extent, if any, clients can view adult pornography 
and/or consume alcohol towards the end of their involvement with the agency. In the author’s 
experience, different jurisdictions have different policies in these areas. 

e Sexual arousal/interest patterns do start to become more clearly established during this time. 
Consideration for the assessment process will include the use of a viewing time measure such as 
the LOOK Assessment or Abel Assessment of Sexual Interest. Although the author uses the LOOK 
(https://www.lookassessment.com/) in his practice, there can be pros and cons to each, and 
DCYF will want to consider the finer points, including cost and clinical utility. 

o For clients with strong abuse-related sexual interests, DCYF will wish to consider the use 
of specialized behavioral treatments to help these clients manage their thoughts, 
fantasies, urges, and behaviors. Although these treatments on their own are not known 
to be effective over the long term, a recent meta-analysis has found that programs that 
contain behavioral treatments appear to produce better outcomes. This is an area in 
which more research is needed. Based on the available research, the overarching focus 
with these clients should be in helping them to manage their behaviors. There is no 
research to date showing that treatment can consistently or reliably change their 
underlying sexual interests. In other words, treatment can’t reliably change what people 
like, but it can help them change their behavior around what they like. All of this said, 
these clients will likely be in the minority of DCYF’s population; the point is that sexual 
interests become more entrenched in adulthood. 

e Depending on circumstances, adult risk assessment measures such as the Static-99r, Stable- 
2007, Acute-2007, and SOTIPS become more appropriate for these young adults, and given the 
numbers of developmentally challenged clients served, DCYF may also want to consider the 
ARMADILO (http://www.armidilo.net/). One of that tool’s authors, James Haaven, is based in 
Oregon. 

e The programs’ treatment approaches, such as ART, DBT, and (possibly) TARGET can be used with 
adults as well as adolescents, with some modification. Likewise, Motivational Interviewing can 
be used with clients of all ages. 

e Abroader approach of trauma-informed care will also be important, as described earlier. 


| hope this overview and suggestions are useful to the competent and committed staff of the DCYF 
programs for youth who have sexually abused. 


Respectfully submitted, 


Davie S. Prescott, LCSW, LICSW (signed electronically) 


David S. Prescott, LCSW, LICSW 
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Introduction 


The State of Washington has long been concerned with appropriate treatment of justice- 
involved youth. In 1999, Washington State’s Juvenile Rehabilitation (JR) implemented a 
competency-based model of treatment and case management for justice-involved youth, 
focusing on increasing youth accountability, skill development and measuring youth changes in 
skill areas throughout their involvement with JR.* In 2000, an effort was undertaken to further 
define and specify appropriate interventions to be used with youth in residential care and with 
families as the youth return to their home communities. This effort gave birth to the 
development of a research-based Integrated Treatment Model (ITM). The ITM was intended to 
be effective through attending to motivation and engagement of both youth and families, 
adopting a commonly understood language to be utilized throughout the juvenile justice 
continuum, teaching a uniform set of cognitive-behavioral skills, promoting generalization and 
maintenance of positive changes, and providing for ongoing clinical consultation system to 
ensure continuity of interventions and adherence to the model.’ The model is described in 
depth elsewhere. In brief, treatment strategies incorporated into the ITM used in residential 
care were derived from cognitive-behavioral therapy (CBT), particularly Linehan’s Dialectical 
Behavior Therapy (DBT), and Goldstein et al.’s Aggression Replacement Training (ART)?. 
Treatment strategies incorporated in the ITM used in community settings are grounded in 
Alexander & Sexton’s Functional Family Therapy (FFT)*. At the time they were incorporated 
into the ITM, the listed therapeutic approaches were all considered evidence- or research- 
based. 


Codifying the state’s commitment to the use of evidence- and research-based practices, the 
Washington State Legislature passed E2SHB 2536 in 2012, mandating that “prevention and 
intervention services delivered to children and juveniles in the areas of mental health, child 
welfare, and juvenile justice be primarily evidence-based and research-based, and...provided in 
a manner that is culturally competent.”° To aid enactment of the mandate, the Legislature 
directed the Washington State Institute of Public Policy (WSIPP) to conduct standard meta- 
analytic and benefit-cost analyses to classify relevant programs as “evidence-based,” “research- 
based,” and “promising.” Such classifications inform eligibility for state funding.® 


Compared to general treatment for youth in JR, specialized treatment for substance use 
disorders (SUDs) in justice-involved youth has not achieved a comparable level of integration 
and standardization despite having been a focus of JR in Washington State for nearly 35 years.’ 
Unfortunately, there exists no uniform SUD treatment model or functioning oversight 
committee in place to guide the implementation of SUD assessment and treatment in juvenile 
justice settings in the state.” The Substance Use Disorder Oversight Committee was disbanded 
to incorporate into the newly formed Behavior Health Quality Assurance Group. However, this 
did not prove successful in addressing the many needs of SUD services. As such, the SUD 
Oversight Committee was reinstituted. Significant variation exists among SUD treatment 
programs across institutions, creating the potential for treatment content and quality to vary by 
institution and making it difficult to compare and monitor programs.’ A recent study by the 
Washington State Department of Children, Youth & Families (DCYF) examining SUD treatment 


needs in JR concluded that efforts to standardize SUD treatment process and ensure youth are 
receiving the same quality of treatment across institutions could facilitate program evaluation, 
support continuity of services, and assist in improving outcomes for youth.’ 


Such efforts are important because SUD treatment needs are consistently high in justice- 
involved youth in the state. DCYF researchers found that, between 2014 to 2018, roughly two- 
thirds of youth in JR showed a need for SUD treatment each year and such need consistently 
exceeded residential treatment capacity.’ During the examined time frame, only 56 percent of 
youth with identified substance use needs started residential SUD treatment, with an average 
of 185 youth per year not receiving needed residential SUD treatment.’ 


Against this backdrop, DCYF sought the input of a consultant to help examine fidelity of the 
evidence-based interventions incorporated into DCYF’s Integrated Treatment Model for JR as it 
relates to specialized treatment for substance abuse. The consultant was contracted to conduct 
an assessment of the quality of treatment that currently exists in JR by visiting at least two of 
the three juvenile rehabilitation institutions, one community facility, and one parole office. The 
assessment was required to address the following questions as they are related to the 
Specialized Treatment for Substance Abuse in JR: 


1. Are youth being appropriately matched with treatment based on need? 
a. What assessment is being used for program eligibility? 
b. Is the assessment being used appropriately? 

2. Is the treatment high quality? 

How well is JR implementing this treatment? 

Are youth receiving the appropriate dosage? 

Do staff receive the correct type and amount of training? 

Does there seem to be variation by location? (between institutions, between 

institutions and community facilities) 

3. How well is the treatment integrated with the assessments, other treatments, and 
reentry planning? 

4. Does the current quality assurance plan adequately measure the treatment model? 

5. Do you see any evidence that would concern you that treatment access varies by race, 
ethnicity, age, or gender? 

6. What recommendations do you have to improve implementation given the current level 
of resources? 

7. What additional resources would you recommend to improve treatment quality and 
integration? 

8. DCYF will be serving youth up to age 25 in the future. What advice would you give the 
agency as they prepare to provide this treatment for older youth? 
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Method 


The consultant conducted site visits at the following facilities: Green Hill School (GHS) on 


December 19, 2019, Naselle Youth Camp (NYC) on December 20, 2019, Echo Glen Children’s 
Center (EGCC) on January 2, 2020, and Woodinville Community Facility (WCF) on January 10, 
2020. Each site visit lasted 2-4 hours, during which time facility staff were interviewed 
regarding the relevant topics. In addition, the consultant spoke with six program managers 
representing the parole regions across the state during their monthly meeting (via phone, 
January 16, 2020). 


Findings 


As described in a November 2019 report by DCYF staff, entitled Residential Substance Use 
Treatment Access in Juvenile Rehabilitation in Washington State’, there is substantial variation 
between institutions in terms of the SUD treatment provided. However, SUD assessment 
practices are uniform. 


Assessment Practices 


All youth are administered the Global Appraisal of Individual Needs—Short Screener (GAIN-SS) 
upon intake. The GAIN SS is a short (15-item) version of the Global Appraisal of Individual 
Needs, developed by Chestnut Health Systems to be used as a screening tool.® It takes 
approximately 5 minutes to administer, consisting of three subscales: the External Disorder 
Screener, the Internal Disorder Screener, and the Substance Disorder Screener (SDS). Each 
subscale has a maximum score of five for a total maximum score of 15. Those who screen 
positive on the SDS are administered a longer assessment, called the Adolescent Substance Use 
Assessment (ASUA), designed by JR to identify the level of severity of use for each substance 
used. The ASUA is adapted from the Adolescent Chemical Dependence Assessment developed 
by JR in 2003 and spans the six dimensions of American Society of Addiction Medicine (ASAM) 
criteria for SUD treatment. Administered by a certified substance use disorder professional 
(SUDP), the ASUA takes one to three hours to complete and produces a score that is reflective 
of the level of SUD treatment needed by the assessed youth. 


Treatment Practices 


Each of the three institutions has its own model of care. 
Naselle Youth Camp 


NYC provides level one outpatient services, which involves individual counseling sessions once 
weekly for about 30 minutes, focused on 1-2 issues. Open group treatment sessions are 
provided a few times per week. With open groups, group sessions are ongoing with youth 
cycling in and out of the groups at staggered times. Alcoholics Anonymous meetings are offered 
twice per month. The youth’s level of severity of SUD has little impact on the form or amount of 
treatment received. The model of treatment provided at the facility generally follows the 
Matrix Model? and the Adolescent Community Reinforcement Approach (ACRA-ACC) with an 
emphasis on education and relapse prevention. Staff noted that the Matrix Model was not 
originally designed for use in adolescent correctional facilities. The model has now been 
adapted by the original developers for a variety of correctional settings including jails, 


community corrections, probation programs, drug courts, and prisons with the structure of an 
evidence-based treatment experience and combines education on both substance use and 
criminal thinking and behaviors. 


At the time of the site visit, no clinical supervision was being provided and no formal 
procedures were in place to promote fidelity to a standardized SUD treatment model. Although 
JR clinical supervisors have been trained in the methods and standards for clinical supervision 
provided by the Matrix Model and ACRA-ACC, clinical supervision had not yet been 
implemented. It was noted that NYC used to have an intensive outpatient program, but that 
was discontinued when the population declined. Currently there are not enough staff to offer 
more intensive services. Staff expressed a desire for a more appropriate standardized SUD 
treatment curriculum and more CD staff, possibly embedded in the living units with protected 
time and higher pay. 


Green Hill School 


GHS provides level two intensive outpatient services (ASAM level 2.1), consisting of half-day 
closed group sessions for 10 weeks (similar to an academic quarter), called the SMART 
program. Cohorts of 8 youth begin and “graduate” from the 10-week groups together. Due to 
the closed nature of the groups, when a youth is expelled due to behavioral issues, their spot 
remains vacant unless it happens in the first week or two of the quarter. Care is taken not to 
have youth from rival gangs in the same group. GHS models the SMART program after the 
Matrix Model, in which staff had received specialized training, but the model has had to be 
adapted to the practical realities of the setting. A subset of DBT skills have been adapted for use 
in the curriculum as well. Staff expressed interest in examining other treatment curricula that 
were more appropriate to correctional settings (e.g., A New Direction or the Matrix Model for 
Criminal Justice Settings, offered by Hazelden Publishing,) but noted no funds were available to 
purchase commercial curricula. The SMART program is manualized so that different cohorts of 
youth have a similar treatment experience. However, there is no clinical supervision and no 
formal procedures in place to promote fidelity besides manualization and “checking boxes.” 
Staff noted there used to be peer audits, which were viewed as helpful but were no longer 
occurring. 


Due to short sentences, some youth identified as needing SUD treatment are unable to be 
enrolled in groups. Such youth are given a relapse prevention workbook and met with 
individually as time allows. At the time of the site visit, GHS had recently hired a SUDP-T for 
level one opioid use disorder treatment (OUD), funded by a time-limited opioid grant. There 
was a desire to provide this level of care for other substances. There was also a desire fora 
dedicated SUD unit to provide inpatient residential treatment. Staff noted that, because 
marijuana use is common in the living units, it has the potential to undermine youth progress in 
the program. Inpatient treatment was viewed as a potential solution to this challenge. 


Youth who have a positive urine toxicology for marijuana are directed to watch a video, called 
“Marijuana and the Teenage Brain” and to complete a self-assessment of consequences of 
marijuana use corresponding to the potential effects identified in the video, akin to a 


motivational enhancement intervention. To the extent possible, staff have been tracking 
follow-up toxicology screens and finding low rates of repeat positives. This seems to suggest a 
benefit of the practice, but it has not been systematically evaluated. The evaluator noted the 
existence of the research-based intervention, the Teen Marijuana Check-up.?° Staff were not 
aware of this intervention but expressed interest in learning about it. 


Echo Glen Children’s Center 


EGCC provides level three intensive inpatient treatment (ASAM level 3.7), called the Exodus 
program, following a model of DBT for substance use disorders (DBT-S). Groups are conducted 
daily and individual sessions occur on a weekly basis according to a comprehensive 
individualized treatment plan. Length of stay depends on individual needs and progress in the 
treatment plan, with the average stay lasting 3-6 months. With this level of care, each group 
session requires the involvement of 3 staff members. EGCC has generally also offered intensive 
outpatient treatment, but at the time of the site visit, this level of care was not being offered 
due to staffing issues. Although EGCC houses males and females, inpatient treatment is 
provided for males only in the Kalama living unit, functioning as its own licensed agency within 
the institution. Because inpatient treatment is provided for males only, while intensive 
outpatient treatment is not being offered, it means no SUD treatment is being provided to 
female youth. Because suicidality and aggressivity take precedence and are treated in the living 
units, youth with comorbid SUDs and suicidality and/or aggressivity do not receive SUD 
treatment. Furthermore, some youth who did not have the high level of severity requiring 
intensive inpatient treatment were nonetheless receiving it because that was the only level of 
care offered for SUDs other than OUD. Level one care for OUD was being provided with time- 
limited opioid grant funding, which was viewed as a major benefit. The opioid grant has 
enabled following youth into the community, and this possibility was desired of other types of 
SUDs. 


Woodinville Community Facility 


Community facilities no longer provide in-house SUD treatment and variation exists in the level 
of care available in the communities surrounding community facilities. WCF is something of an 
exception as it embeds a staff member of an outside agency in the facility to provide SUD 
treatment, covered by state insurance. This agency uses the Seven Challenges outpatient 
model of treatment, which is manualized and uses standard workbooks (one workbook per 
challenge). The agency staff member providing the treatment has received specialized training 
from the Seven Challenges intervention developer and attends booster training sessions yearly 
to maintain fidelity to the model. The agency staff member has also received training in DBT 
and DBT-S in addition to training in leadership and teaching. Facility staff noted they receive 
annual training in suicide prevention and community safety but not in alcohol or drugs. 
Substance use was described as the one re-entry domain in which staff had the least training. 
The treatment provider is not included in Re-entry Team Meetings. 


Youth entering the facility from one of the institutions have their GAIN-SS results documented 
in the Automatic Client Tracking (ACT) system, which is accessible to WCF staff. Those who had 


screened positive for SUD are referred to the agency staff for SUD assessment and treatment. 
The provider conducts a full ASAM assessment. As SUD treatment records do not travel with 
the youth from the institution to the community facility, the provider relies on the youth to say 
what they had been working on in their prior treatment. Youth are taken to an outpatient 
setting for the treatment, which consists of 12 weekly open group sessions and 12 individual 
sessions. Family sessions are conducted monthly or as needed. Family sessions are advisable to 
the extent that they were included in the protocol of the research conducted on the 
effectiveness of the Seven Challenges treatment approach. Youth are subjected to random 
urine toxicology screens. A satisfaction survey is administered to youth at the end of treatment. 
Responses have reportedly been consistently positive. 


Parole 


Youth being transitioned out of institutions or community facilities may be released to parole. 
When appropriate, parole is included in re-entry team meetings a month prior to release, 
where they typically learn about a youth’s SUD diagnosis. Information about substance use is 
also found in the Initial Client Information to which parole has access. SUD treatment may be 
stipulated as a condition of parole. SUD treatment provision varies by state region and type of 
parole. For example, in King County, SUD treatment is generally provided by Ryther Child 
Center. In Snohomish County, it was noted there was a recent change in practice; after using 
an OUD treatment navigator to help youth with OUD find appropriate treatment, treatment 
navigation services were provided for other SUDs as well. Not all regions have treatment 
navigators, however. In some cases, a mental health coordinator helps to find SUD treatment 
options. Parole program managers noted that clients often lack motivation in getting help. They 
felt some sort of incentivization would be helpful, and there was interest in Contingency 
Management though this evidence-based practice was not being used. Desired incentives 
included housing support, gym memberships, and clothing. It was noted that CDPs were 
embedded in parole offices in the 1990s to serve as drug and alcohol coordinators, and this was 
felt to be beneficial. Program managers stated that some youth fall through the cracks as they 
are waiting to get an assessment in the community or waiting to secure an available bed in a 
treatment facility. Because data on SUD treatment in the community are not captured, 
treatment referrals and admission activities are not monitored consistently throughout the 
system. 


Analysis 


Are youth being appropriately matched with treatment based on need? 


It does not appear that youth are being appropriately matched with treatment based on need. 
While there is a range of levels of care provided across the institutions, each institution only 
provides one or two levels of care that may be dependent on what substance is being used. 
Assignment of youth to particular institutions is made based on multiple considerations, 
including age, gender, sentence length and other considerations, of which SUD 
prevalence/severity is not one. Youth in need of SUD treatment get the level of treatment 
offered at the institution to which they were remanded, regardless of their level of 


need/severity. In many cases, this means youth get less treatment than they need, in some 
cases, more treatment than they need, and in other cases, no treatment at all. 


What assessment is being used for program eligibility? Is it being used appropriately? 


The GAIN-SS and the ASUA are being used to determine program eligibility. They appear to be 
administered appropriately, but the results with regard to severity are not fully being taken into 
consideration due to level of care constraints at the institutions to which they are remanded, as 
noted above. Though modeled after a validated assessment, the ASUA is not itself validated. 
The ASUA was not reviewed during the course of this evaluation. 


Is the treatment high quality? 


In general, the treatment models selected by the different programs are research-based and of 
high quality. However, some of the treatments used were not developed for juvenile justice 
settings and have had to be adapted considerably. When a treatment is adapted and applied to 
a different population that the one in which it was originally tested and found to be effective, it 
is arguable whether it can still be called evidence- or research-based. Thus, staff should be 
dissuaded from adding outside materials without prior vetting by clinical experts or an 
appropriate oversight body. 


How well is JR implementing this treatment? 


JR appears to be implementing the chosen treatments as well as possible given the constraints 
of the settings. Treatments are generally manualized and delivered in the context of an 
individualized treatment plan. Staff are passionate about providing high quality SUD care. 
However, systemic issues interfere with the ability of JR to provide optimal treatment. For 
example, it was noted that staffing was often a problem. As noted, at the time of the site visit, 
EGCC was not providing intensive outpatient due to staffing challenges. As long as this is the 
case, no girls in a JR institution receive SUD treatment. With three staff members needed to 
conduct each intensive inpatient group, when one staff member is pulled into another unit by 
management to cover a staffing shortage, it means that youth do not receive treatment that 
day. Staff noted that, compared to those working in the living units, those working in SUD units 
require substantial additional specialized training, yet they do not have differential titles or pay, 
and their time conducting SUD treatment is not protected. This also affects staff turnover. 
Such systemic issues harm the quality of SUD care offered to youth in JR. 


Are youth receiving the appropriate dosage? 


In many cases, the answer is no. As noted above, youth receive the type of treatment offered 
at the institution to which they have been remanded, regardless of severity of need. In some 
cases, treatment is a fixed dosage (e.g., 10 weeks at GHS), and in other cases dosage is more 
flexible (e.g., inpatient stay length at EGCC). Sentence length sometimes precludes receiving 
the appropriate dosage of treatment, making referral to appropriate ongoing care even more 
critical. Left to receive treatment in the community, youth sometimes fall through the cracks as 
they wait for a spot in a community-based program. 


Do staff receive the correct type and amount of training? 


For the most part, institution staff appear to be appropriately trained in the treatments that 
they are providing. However, in community facilities it was noted that substance use is the one 
re-entry area in which they receive the least training. 


Does there seem to be variation in treatment quality by location? 


Treatment quality varies primarily according to level of care. Higher levels of care are generally 
of higher quality because they allow more intensive contact with youth receiving services. As 
noted, the institutions vary according to levels of care provided. There is variation in treatment 
quality between institutions and community facilities and likely also between community 
facilities as community facilities have outsourced their SUD treatment provision. While the 
treatment provided through WCF is judged to be high quality, other community facilities may 
not have access to such high-quality treatment in their communities. 


How well is the treatment integrated with the assessments, other treatments, and reentry 
planning? 


SUD treatment is not well integrated with SUD assessment. SUD treatment is provided largely 
without regard to the level of severity identified by the assessment. SUD treatment does not 
appear to be well-integrated with mental health treatment. It appeared that degree of 
communication and coordination between SUD treatment providers and mental health 
providers varied by location. SUD treatment was also not well integrated with the treatment 
provided in the milieu. It does not appear that residential counselors support SUD treatment in 
a consistent or meaningful way. 


Does the current quality assurance plan adequately measure the treatment model? 


There was no formal quality assurance plan identified at any of the sites visited. Fidelity to the 
chosen treatment model was supported through manualizing the treatment and/or using 
standard curricula, workbooks, and/or handouts. Much more could be done to promote quality 
assurance, such as re-instating peer review, providing regular consultation with treatment 
model trainers/developers, or providing technical assistance in the form of clinical supervision 
or learning collaboratives. To ensure a successful quality assurance plan, it would be advisable 
to secure buy-in from leadership at the institutions that they will support the plan once it is 
implemented and refrain from deviating from established curricula without appropriate vetting. 


Do you see any evidence that would concern you that treatment access varies by race, 
ethnicity, age, or gender? 


There was no evidence that treatment access varies by race, ethnicity, or age. There was clear 
evidence that treatment access varies by gender, with the only institution serving girls 
providing only inpatient treatment to boys. This appeared to be a temporary situation though 
it was very concerning. 
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What recommendations do you have to improve implementation given the current level of 
resources? 


Institution staff expressed a desire for better communication with and support from 
behavioral health administrators. 

Short of providing SUD staff with higher pay, providing SUD treatment staff with a new 
title that reflects their different role within the institutions could discourage 
management from pulling them out of SUD care to cover for other types of staff. 

It may be possible to receive technical assistance from federally funded sources, such as 
the Addiction Technology Transfer Network. 

Re-instating peer review could also improve implementation. 

Continuity of care could be enhanced by providing SUD treatment records to community 
facilities when youth arrive there from institutions and to community treatment 
providers with whom paroled youth continue their treatment. This would also save time 
and cost associated with re-assessment. 


What additional resources would you recommend to improve treatment quality and 
integration? 


Make regular use of an SUD treatment oversight committee to guide the 
implementation of SUD assessment and treatment in juvenile justice settings in the 
state. 

Increase the correspondence between severity of SUD identified via assessment and the 
level of treatment provided. 

If it is not possible to make institutional assignments on the basis of level of treatment 
need, each institution should provide multiple levels of care to better match youth to 
SUD treatment according to need. This would clearly require increased staffing and 
expanded programming. 

Reinstating SUD treatment coordinator positions would be beneficial. 

Additional staffing could provide suicidal and aggressive youth with SUD treatment as 
their suicidality and aggressivity is concurrently addressed. 

The problem of released youth falling through the cracks could be addressed by 
providing for youth to be followed into the community, as currently happens for youth 
with OUD under the opioid grant. Staff at Echo Glen were particularly in favor of this 
and seemed to have ideas about what this might look like. For example, it was felt that 
Echo Glen could serve as a hub for the state due to its relatively central location. 
Funding for alternative curricula could also improve treatment quality if other 
treatments can be identified that better fit the constraints of correctional settings. 
Refresher training for providers in their treatment models could enhance fidelity. 
Curriculum review by credentialed treatment trainers could also be beneficial. 

There is a potential benefit of providing prevention education for all types of SUDs as is 
being provided for OUD. Marijuana use disorder prevention programming would be 
particularly beneficial given the increasing prevalence of marijuana use. The research- 
based Teen Marijuana Check-up would be advisable to examine as a possible 
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intervention strategy.*° 

To whatever extent possible, efforts should be undertaken to promote coordination 
among SUD, mental health, and milieu treatment providers. 42 CFR Part 2 creates real 
barriers to coordination and continuity of care. 

o Could releases of information from or on behalf of youths be procured to 
facilitate communication among care providers? If so, beyond promoting 
communication among SUD, mental health, and milieu counselors, it would 
potentially be beneficial to include SUD treatment providers in re-entry team 
meetings. 

o If not, could one-way communication mechanisms be implemented where SUD 
treatment staff are made aware of any mental health treatment being provided? 

o Inlight of the high rates of SUDs and SUD treatment need in the general 
population of youth in JR, consider the utility of treating every youth as though 
he or she has an SUD. This “SUD-informed approach” could be analogous to a 
trauma-informed approach wherein all clients are treated as though they have 
histories of trauma. With respect to SUDs, this could involve bringing residential 
counselors into the loop regarding what youth in general are working on in SUD 
treatment. With that awareness, residential counselors could perhaps support 
that work by modeling such skills for all youth in the milieu. If this is not feasible, 
consider what an “SUD-informed approach” in the milieu might look like and 
whether it would look different from what approaches currently being used. 


What advice would you give DCYF as they prepare to provide treatment for youth up to age 


25? 


Treatments that are appropriate for adolescents are not necessarily appropriate for 
young adults and vice versa. Care must be taken to ensure treatment is developmentally 
appropriate. 

Similarly, screenings and assessments that have been validated and are appropriate for 
one age group are not necessarily valid or appropriate for a different age group. The 
GAIN-SS has been validated in both adolescents and adults.‘ The ASUA should be 
validated for both adolescents and young adults. 

With higher ages, sentences are likely to be longer. With long sentences, the question 
of timing of services should be addressed. Will programs prioritize providing SUD 
treatment when SUDs are at their worst (upon intake) or relapse prevention 
programming as youth face release from a long stint in a controlled environment? 
Among youth, rates of marijuana use in the community increase with increasing age. ‘* 
As older youth will be served, there may be an exacerbation of marijuana use in living 
units in both institutions and community facilities. How this will be handled should be 
considered. 
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Author Note 


Susan Stoner, Ph.D., is a licensed psychologist and Research Scientist at the Alcohol and Drug 
Abuse Institute at the University of Washington. In 2015, she was tasked by the Washington 
State Division of Behavioral Health and Recovery (DBHR) to review the literature and prepare a 
report on evidence-based, research-based, and promising practices for adolescent substance 
use disorders.4 In 2018, Dr. Stoner prepared a report for DBHR on evidence-based practices in 
behavioral health in Washington State.?° 
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JUVENILE REHABILITATION MENTAL HEALTH 
TREATMENT QUALITY ASSESSMENT 


Are youth being appropriately matched with treatment based on need? 


Mixed. The Beck Depression Inventory (BDI) and UCLA PTSD Reaction Index (RI) 
appear to be consistently used across institutions upon a youth’s first arrival at a facility. These 
are validated and widely used measures of mental health need and are appropriate for the age 
group served by JR. Both measures are appropriate for teens and adults. Universal screening 
using these tools allows the institutions to identify mental health needs that may otherwise not be 
reported or referred. This is a strength of the mental health treatment programs (Woodinville is 
not expected to perform independent mental health screening as they receive the youth’s medical 
files when a youth is transferred). However, information about the use of screening and 
assessment information to guide specific treatment was available inconsistently across 
institutions and across diagnostic types. 


Following referral, treatment plans are typically developed without an explicit expectation that 
treatment will follow clinical guidelines based on areas of elevated need identified by the 
assessments. My review did not include review of medical records and I cannot say how often 
treatment aligned with best practices, but policies were not in place to ensure therapists were 
aware of and expected to follow guidelines when delivering treatment. Trauma treatment at Echo 
Glen appears to be the only therapeutic program where an elevated score on a mental health scale 
(UCLA) leads to a specific treatment recommendation (Trauma-Focused Cognitive Behavioral 
Therapy). In addition, scales were not used to monitor the progress of treatment over time. Only 
Green Hill was routinely reassessing youth using the BDI and UCLA scales and this was at 
release and not incorporated in active treatment. One of the primary benefits of using symptom 
measures when developing treatment plans is to reassess progress at routine intervals and adjust 
the therapeutic approach, if necessary. This approach (measurement-based care) did not appear 
to be a systematic part of clinical care at any of the facilities. 


Is the treatment high quality? 


Mixed. In this report, I interpret high quality as the explicit expectation and use of 
evidence-based clinical elements in psychotherapy treatment and a strategy for adjusting 
treatment when symptoms are not improving. I did not assess psychotropic prescription 
practices, use of polypharmacy or other medication management. In addition, I did not observe 
any psychotherapy sessions or review medical records. I base my assessment off of staff report 
of the use of effective clinical interventions. All of the mental health staff I met with reported the 
use of at least some effective clinical elements of treatment: Exposure for social anxiety (Green 
Hill; Echo Glen), cognitive-behavioral therapy for trauma (Echo Glen), cognitive behavioral 
therapy for depression (Echo Glen), Cognitive Behavioral Intervention for Trauma in Schools 
(Naselle, Echo Glen). Apart from the psychiatry fellows rotating through Echo Glen, it was not 


clear what level of training the mental health treatment providers had or were receiving to stay 
up to date with best practices. Mental health treatment at Echo Glen is highly specialized with 
youth receiving only trauma-focused treatment from the staff psychologist and other treatment 
needs (e.g., anxiety, depression, eating) were referred to the psychiatry residents. At Green Hill 
and Naselle, all types of psychosocial treatment were delivered by one staff member (both 
master level clinicians) who needed to be able to address a wide range of presenting needs. 
Variation in how mental health treatment occurs in the sites also reflects a lack of shared clinical 
oversight. Individual treatment in Naselle is typically short term (5 sessions) while treatment for 
trauma focused CBT is the standard length required by the curriculum (about 12 weeks), and all 
other mental health treatment at Echo Glen and Green Hill is at the youth’s discretion and can 
occur throughout their entire stay. 


While all of the mental health treatment providers were clearly motivated, compassionate and 
conscientious, they also all reflected on the inadequacy of the system (lack of sufficient staff, 
high complexity of needs) for delivering high quality mental health treatment. Neither approach 
(short term or through the entire term) appeared to be guided by symptom reduction (with the 
exception of TF-CBT which could be extended if youth symptoms were not resolving). Rather, 
much of long term treatment appear to be focused on providing ongoing, supportive care rather 
than targeted symptom reduction. Many youth appear motivated to engage in this type of therapy 
(i.e., more integrative therapy in which youth are focused on understanding and processing past 
trauma and formative events) and find it helpful. However, the evidence base suggests that 
shorter term, more symptom focused and directive treatments are often superior to less directive 
treatment in reducing symptoms. 


It is also complicated to set symptom reduction as a criterion for delivering high quality care for 
JR in that being incarcerated is an active stressor and the interviews revealed that many youth are 
seeking guidance for how to cope with being in the facility and experiencing anxiety about what 
to do upon release. It could be that release from incarceration would be the fastest way to 
support symptom reduction for a number of youth. Barring this, general coping with active 
stressors could be addressed more manageably through group therapy focused on addressing 
common stressors of incarceration, and common fears about release. Ideally, this would continue 
to be facilitated by trained mental health providers in order to weave effective clinical skills into 
the group sessions. This could free up time for therapists to be focusing on one on one treatment 
for youth who have symptoms that are actively impacting their ability to function within the 
institution. Another complicating factor for recommending treatment within institutions is that 
the most effective treatments for complex mental health needs (ADHD, Autism/DD, Bipolar, 
psychosis) involve coaching individuals in the youth’s immediate environment (family, schools) 
to promote more positive adaptation and functioning. Consequently, gains achieved during 
residential stay may not transfer with the youth to environments outside of the facility where 
families and schools would then need to be engaged into treatment to sustain improvements. 


How well is JR implementing this treatment 
Mixed. The only treatment well-specified enough to evaluate for implementation is TF- 


CBT and, in this case, appears to be well-implemented with a highly qualified treatment provider 
(Echo Glen) who is actively engaged in the trauma treatment community. Otherwise, treatment 


occurring in the institutions and being accessed in the community by youth in the community 
group home is not well-specified enough to evaluate. 


Are youth receiving the appropriate dosage? 


Unable to Assess. As there is no uniform reporting system for youth mental health 
outcomes through repeated measurement of symptoms, there is no way to robustly assess 
whether youth are receiving the appropriate dosage of treatment. The report of current practices 
suggests, however, that youth are likely either receiving not enough or more than necessary. 


Do staff receive the correct type and amount of training? 


Not Acceptable. Mental health therapists serving youth in the JR institutions are not 
being routinely trained in best practice clinical treatment standards. The fellowship rotation at 
Echo Glen ensures a number of youth have access to treatment by residents who are being 
exposed to evidence-based treatment strategies through other rotations but training for JR 
therapists is otherwise limited to what the clinician was trained to do prior to hire and whatever 
clinical strategies they are motivated to learn more about as part of their ongoing clinical 
education. The treatment approach is generally eclectic and therapist are being called upon to 
address a very wide range of needs without adequate training and consultation support. 


How well is the treatment integrated with the assessments, other treatments, and 
reentry planning ? 


Acceptable. A strength of mental health treatment across all institutions is the effort to 
coordinate the medical and psychosocial treatment of mental health needs through team-based 
planning and ongoing coordination. Treatment plans are reviewed with mental health therapists, 
coordinators, psychiatrists and medical directors in the three institutions. Mental health service 
coordination between institutions and in the community when youth transfer is a significant 
focus but challenged by the lack of appropriate treatment in the community and disruptions in 
the continuity of treatment. A notable achievement is the system-wide focus on reactivating 
Medcaid eligibility for youth leaving JR care and working with local wraparound (WISe) teams 
in the community to support reentry. Many of the challenges of service coordination require 
coordination with systems outside of JR control but still present opportunities for JR to 
reconsider some clinical and coordination strategies. A critical area of need is ensuring 
continuity of care during transfers to different facilities within JR and during community release. 
Institution mental health therapists raised concerns about youth who are transferred to group 
homes no feasible way to continue care. Telemedicine facilities are inadequate, largely because 
offices with video chat functionality are also administrative offices that youth cannot be left in 
unsupervised. The group home that was part of this interview (Woodinville) went to significant 
lengths to ensure youth with mental health needs were connected with local community 
providers but the quality of these providers varies widely and the group home does not have 
control over the quality and availability of treatment. As outlined in more detail in the 
Woodinville summary, this has led to seriously concerning misdiagnoses and medication 
changes that resulted in youth being sent back to institutions for stabilization. A serious concern 
for reentry is the lack of follow up regarding whether a youth reenrolled in Medicaid, connected 


to local a mental health provider through a WISe team or individual provider and whether a 
youth obtained prescribed medication. This is another way that being committed to JR can 
actively worsen mental health status as eligibility for Medicaid is terminated once a youth is 
committed and they must reenroll or have benefits unsuspended in order to continue to receive 
services at release. If this does not go smoothly, youth are returned to community with fewer 
medical resources than they had before commitment, particularly if they were receiving mental 
health services prior to sentencing. 


Does the current quality assurance plan adequately measure the treatment model? 


Not Acceptable. Quality assurance support for mental health treatment at JR appears to 
be primarily focused on strategies for initial mental health assessment, service coordination and 
for the milieu treatment (integrated treatment model) that was not part of this review. There did 
not appear to be a quality assurance plan for monitoring the treatment of specific mental health 
disorders. 


Do you see any evidence that would concern you that treatment access varies by 
race, ethnicity, age, or gender? 


Mixed. I did not see any evidence that treatment was not systematically offered to youth 
on the basis of demographic status with one exception, substance abuse services are provided in 
a cottage unit at Echo Glen that is male-only. However, I also did not see evidence that mental 
health treatment was explicitly mindful of race/ethnicity and cultural considerations in training 
being provided to therapists or in models being used in treatment. For example, racism is 
increasingly being recognized as a stressor that can heighten symptoms of trauma and using 
culturally-grounded metaphors and cultural traditions to adapt treatment can engage better buy in 
and treatment outcomes. This would be extraordinarily difficult for the individual therapists to 
manage adequately on their own, along with being expected to treat such a wide array of mental 
health needs. 


What recommendations do you have to improve implementation given the current 
level of resources? 


Many of my recommendations are embedded in my comments above and I summarize them here 
in order of priority based on what I think would best support youth mental health stabilization 
and long term functioning: 


1. Restructure the delivery of mental health treatment to provide more stepped care 
approaches beginning with group-based treatment facilitated by mental health staff using 
evidence-informed curricula for treating anxiety, depression, trauma, and social skills 
(particularly for youth with ADHD, not just focused on anger management principles) and then 
moving to one on one treatment for youth who continue to display consistent and concerning 
behaviors (self harm, externalizing). Use an electronic platform for measurement-based care 
during individual treatment to track specific behavioral targets and use clinical supervision to 
adjust clinical approaches as needed. Ensure youth have access to general, non-clinical social 
support resources (e.g., coaching) for reinforcing and maintaining good problem-solving skills, 


positive self-identity, within the cottages but do not expect line staff to deliver sophisticated 
clinical models. Adopt standard treatment guidelines for use across all facilities and consider 
using mental health coordinators as liaisons to educate and encourage the use of these curricula 
in community treatment clinics in areas surrounding group homes (e.g., co-training). 


2. Improve clinical continuity by either a) involving regional mental health coordinators 
from the youth’s home community in their treatment planning immediately after placement and 
have the coordinators act as advocates/monitors for quality and continuity of care throughout the 
youth’s various placement transfers and reentry to community; or b) improve telemedicine 
facilities and consider other ways to leverage mobile health or digital platforms so that youth can 
connect with institution mental health treatment providers to avoid gaps in care as youth are 
enrolling in community services. 


3. Ensure mental health treatment plans and targets of therapy are integrated in 
milieu behavioral goals and planning by adapting current family system models to work with line 
staff in cottages and units. Expand the use of family-based interventions for youth at reentry 
(e.g., the work that will occur with the NIMH grant under the supervision of Dr. Ahrens and Ted 
Ryle). As much as possible, look for ways to support local communities to deliver these models 
instead of institutional placement through Option B or other sentencing alternatives, particularly 
for younger youth and those with autism, ADHD, developmental delays and learning disabilities. 


What additional resources would you recommend to improve treatment quality and 
integration? 


Mental health treatment staff capacity needs to be expanded but it should be done in 
concert with some restructuring of services, e.g., more stepped care approaches, in order to 
ensure services are effective and efficient. Mental health clinical quality oversight needs to be 
instituted with more resources invested in cross-site training and clinically-oriented 
supervision/consultation on cases. JR cannot ensure the continuity of care in reentry alone. The 
mental health and child welfare systems need to coordinate to adequately fund transition 
services. An urgent need is a position to monitor the mental health service continuity of all youth 
at reentry to ensure Medicaid enrollment, service enrollment and economic/housing stability 
(note: the hiring of a housing navigator occurred shortly after my site interviews) and to support 
the quality of service at the community level. 


JR will be serving youth up to age 25 in the future. What advice would you give the 
agency as they prepare to provide this treatment for older youth? 


In many ways, institutional placement can more effectively accommodate the treatment 
of adult mental health needs than youth mental health needs. Mental health treatment staff should 
be trained in the developmental tasks of emerging adulthood and brain development. They 
should be provided with more information on prodromal and first episode psychosis 
identification and treatment strategies. Substance misuse is likely to be an even more 
predominant issue and treatment for SUD will likely to need to expand. Reentry services will 
need to focus even more on housing and employment and in the case of youth adults with serious 
mental illness, should use evidence-based approaches (e.g, supportive employment). 


Please provide a statement of qualifications 


I have a PhD In Counseling Psychology from the University of Southern California, a clinical 
degree for which I was trained to competently deliver psychological assessments and 
psychotherapy for a range of mental health disorders. I currently direct a research and policy 
center that serves as a resource to Washington State on the use of effective clinical treatments for 
child and youth mental health (Evidence-Based Practice Institute). I have fifteen years of 
experience as a researcher and program developer in the area of youth behavioral health. My 
evaluation and recommendations in this report are my own and in no way represent the views of 
my employer (University of Washington). 


Woodinville Group Home 


The Woodinville Group Home is a 15 bed, step down facility for youth transitioning from larger 
institutions to release. It is a minimum security facility and youth must abide by the facility rules and 
expectations in order to remain at the placement. Youth at Woodinville are expected to engage in school 
and/or employment programs. The facility maintains work relationships with surrounding businesses. The 
group home is not equipped to internally manage mental health treatment; however, a high percentage of 
youth sent to the facility have some level of mental health need. Most often this includes medication 
management but may also include the need for psychosocial treatment. 


The interviews informing this summary were conducted in person with the facility Director and Mental 
Health Regional Coordinator. Mental health services are coordinated with external service providers. 
Providers not included in the interview include Ryther, a multiservice behavioral health agency in 
northeast Seattle; Healthpoint, a Federally Quality Healthcare Center that is a relatively newer service 
provider for Woodinville; Valley Cities, a community mental health center; and Northshore for substance 
dependence treatment. Woodinville youth are additionally connected to Wraparound with Intensive 
Services (WISe) community mental health providers as part of release planning. 


Assessment 


Youth mental health needs are typically identified through the transition packet provided by the institution 
as part of release planning. The packet includes mental health assessment reports and notes any previous 
and current prescription medications and previous psychiatric inpatient stays (CLIP or Seattle Children’s 
facilities, typically). In addition, facility staff may reach out directly through email or a phone call to 
discuss the youth’s case and needs prior to transfer. The interviewees noted that the Naselle is the most 
consistent in providing this information. The interviewees noted the Sexual Aggressive and Vulnerability 
Youth (SAVY) and Suicide and Self Screen (SSS) are included in the packet of youth information. This 
packet of information is forwarded to the youth’s prospective case manager in the group home. Either the 
director or the case manager may speak to the transferring facility to discuss additional, sensitive 
information. The interviewees noted that Naselle is also good at discussing the WISe program with youth 
before transition. 


In addition to the group home review, the regional mental health coordinator maintains a list of all youth 
with identified mental health needs transferring to JR regions 3 and 4, which includes the Woodinville 
group home. She obtains mental health information from the Initial Client Information (ICI) form and the 
WISe eligibility tool to determine which youth follow. If a youth is placed at Echo Glen prior to transfer, 
the mental health coordinator and a representative from Healthcare Authority, the state agency overseeing 
WISe implementation, will visit the youth to discuss the program. The coordinator also sits on two 
regional governance teams for mental health administration, the FSPYRT north and south. 


Treatment 


While the regional mental health coordinator is a significant resource for youth on parole who are also 
eligible for WISe, this leaves youth with mental health not on parole underserved. Further, less than half 
(about 40%) of youth eligible for WISe agree to enroll and, of these, only about 20% will follow up with 
services. One of the challenges is that the local WISe programs must reach out to screen and engage 
youth who are eligible. Some WISe programs are willing to meet or call you prior to release, but capacity 
varies by site. 


Woodinville has a long term treatment partnership with Ryther and, specifically, with Ryther’s adolescent 
psychiatrist. The interviewees felt youth are able to receive high quality mental health care at Ryther 
because of the staff’s expertise and familiarity with the needs of Woodinville’s youth, including expertise 
in Dialectical Behavioral Therapy. In years past, Ryther would send mental health staff to the facility to 
conduct assessments and provide treatment but declining population at the group home make these trips 
inefficient for Ryther. Now the group home staff drive youth for appointments. Ryther is about 15 miles 
away from the facility and transporting and waiting for youth can take about 3 hours of a group home 
staff member’s time. Because of the travel and waitlist, Woodinville is exploring other treatment 
partnerships that are closer or may be able to see youth more quickly. Other agencies include Bothell 
Health point, Valley Cities and Northshore for substance use treatment. 


Communication to Woodinville staff from treatment providers varies by agency. At Valley Cities, the 
counselor will speak to the group home case manager after the session. Ryther general sends a post- 
session generic report regarding medication dosage and treatment plan. Behavioral Health Point will also 
send a treatment plan report. Interviewees generally felt that the connection between treatment and group 
goals were weak. Each youth is assigned a case manager within the group home who meets with the 
youth at least weekly to review the youth’s progress towards goals and support school and employment 
success. Facilities vary in their ability to match youth with a preferred provider based on gender or race. 
Interviewees reported that, at the time of the interviews, their perception was that all of the therapists at 
Northshore were Caucasian. 


Youth with unmet mental health needs impact the social climate at the group home, particularly youth 
who are opting to not take prescribed medication. Occasionally, these youth will return to a larger facility 
where there are more supports for complex needs. A more common or difficult treatment change is 
drug/alcohol use. Youth will report that they received treatment at a larger facility but will still come to 
the group home with dependence and may have been using while in the larger institution. A drug and 
alcohol counselor from Northshore comes to the facility to conduct groups and provide individual 
counseling but Woodinville staff felt that this was not sufficient for some youth. Youth involved in both 
mental health and substance use treatment may get up to three hours of treatment a week. About 60-75% 
of youth come to the facility with a substance use problem. Marijuana, methamphetamine and alcohol are 
the most commonly used substances. The facility does not get as many youth with opioid addiction 
although they do occasionally and will have Narcan available for use. Therapeutic Health Services (THS) 
is the local provider who can administer Vivitrol if needed. 


Characteristics of youth 


The most common mental health symptoms for youth at Woodinville are sleep dysregulation, attention 
span and functioning outside of the facility at school or work. The employer may report that the youth is 
unmotivated or refuses to participate in certain activities. There are not many incidences of reactivity or 
aggression. The most common diagnoses are anxiety, depression, PTSD (which is increasing in 
prevalence) and occasionally psychosis. 


The youngest youth in the group home is 16 and most clients are 18-20. Older youth and young adults 
typically have long sentences and the system is eager to have these youth move into group homes to have 
a more real life experience. Housing after release is a significant issue as most of the youth are adults and 
not planning to return to live with family. A housing coordinator for all of JR is expected to be hired 
within the next year but current resources are limited to supporting these transitions. 


Recommendations 


The interviewees identified some suggestions for improving current treatment and coordination of mental 
health services. These included system level coordination and communication with more time dedicated 
to regional staff communication about clients’ mental health needs, transitions and treatment. Suggestions 
also included having the mental health coordinator and other external support staff join reentry team 
meetings to encourage youth engagement with mental health services after release. Woodinville staff felt 
they could begin implementing this immediately with no additional resources required. The interview also 
identified the need for more resources dedicated to transporting youth to mental health appointments 
and/or developing better facilities for telemedicine. The group home has only had one youth participate in 
telemedicine with the referring state institution. In order to implement, an IT person would be needed for 
consultation and setting up a secure system. Finally, the interviewees noted the importance of hiring 
engaged, enthusiastic group home staff to support youth with medication and treatment compliance. 


Echo Glen 


Echo Glen Children’s Center is a medium/maximum security facility located approximately one hour east 
of Seattle. The population primarily includes young males, males with specific behavioral health 
treatment needs, and females. The facility maintains separate therapeutic units for males with substance 
use disorders and males charged with sex offenses. All youth at Echo Glen receive Dialectical Behavioral 
Therapy group sessions and Aggression Replacement Training group sessions. 


The interviews informing this summary were conducted in person for 2.5 hours with the Staff 
Psychologist, the staff Psychiatrist and the Associate Superintendent. Additional written information was 
provided by Echo Glen’s Medical Director/Adolescent Medicine physician. Other staff involved in 
mental health oversight tor treatment include the Mental Health Coordinator, the JR Medical Director and 
the Health Center Administrator. 


Assessment of mental health needs 


Mental health needs may first be detected in the medical assessment conducted by nursing staff within 
three hours of a youth’s arrival to the facility. This includes an assessment of self-harm risk (Suicide Self 
Harm Scale). Within the first seven days of arriving at Echo Glen youth receive a mental health 
assessment from a licensed, clinical psychologist. The assessment includes the anxiety and depression 
subscales of the Beck Depression inventory, and the UCLA PTSD scales. This information is integrated 
with medical information obtained in the initial nursing screen. Within two weeks, youth also receive the 
Integrated Treatment Assessment (ITA), a comprehensive assessment that includes mental health history, 
use of psychiatric medication, risk for sexual victimization or aggression (SAVY) and suicidal ideation. 
At the present time, the Beck and UCLA are not readministered prior to release but a new masters level 
practicum agreement may provide the capacity to complete these in the future. A youth will be referred 
for a psychiatric evaluation if they have an active prescription for psychiatric mediation, or may be 
referred if they have an eligible score on the SSS scale, significant disruptive behavior as reported by 
detention (or the transferring institution), are referred from the general adolescent medicine physician, 
cottage staff concerned about active mental health symptoms, or are self-referred. 


Youth identified as having a mental health need through any of these pathways will have their case 
discussed at the weekly Mental Health Assessment Team (MHAT) meeting. The MHAT meeting includes 
the attending psychiatrist, nursing staff, the staff psychologist, a facility administrator, cottage program 
managers, and the mental health coordinator. Typically, the psychologist has met with the youth, 
administered the mental health assessment and conducted a clinical interview in time for the MHAT 
meeting immediately following a youth’s arrival. Youth can also be referred for a mental health 
assessment at any time during their stay. 


Treatment 


Three different specialized mental health treatment pathways are available to youth: Medication 
management, general psychotherapy, and trauma-focused psychotherapy. Youth already on medication or 
requesting medication meet with an attending psychiatrist or a psychiatry resident or fellow for 
medication evaluation and follow up management. A psychiatrist aims to follow up within 30 days of 
referral but large caseloads lead to average wait times of 2 months. For psychotherapy referrals, average 
wait times are closer to 4 months. In both cases, youth referred to psychiatry/therapy stay in treatment 
until discharge (average of 8-10 months). Medication management is guided by best practice principles 
for minimizing the use of psychotropics. 


Youth with treatment needs other than trauma will be referred to one of the psychiatry residents or 
fellows for individual therapy. Fellows rotate to Echo Glen from Seattle Children’s Hospital and are 
typically trained to deliver cognitive behavioral therapy for depression and generalized anxiety, exposure 
for social anxiety and dialectical behavior therapy for suicide and emotional dysregulation. As noted by 
the facility psychiatrist, the complexity of diagnosis and mental health presentation is high and, in 
addition to psychotherapy, mental health staff are often frequently called to consult with the individual 
living units about how to support youth. The psychiatry residents are expected to routinely assess youth 
using validated mental health symptoms measures (PHQ-9 and GAD-7) to monitor treatment progress. 
Youth typically stay in active treatment throughout their time at Echo, leading to longer wait times for 
new cases. 


Youth diagnosed with Post Traumatic Stress Disorder or Trauma Disorder NOS receive Trauma-Focused 
Cognitive Behavioral (TF-CBT) treatment from the staff psychologist. Trauma experiences are common 
among youth at Echo Glen; staff estimated that 100% of youth screened with the UCLA trauma screen 
since 2017 have reported at least one trauma experience and an average of six traumatic experiences. 
TFCBT treatment lasts 16 weeks to 20 weeks (for complex trauma) and the average length of treatment is 
5 months. Treatment goals are centered on symptom reduction so that symptoms do not interfere with the 
youth’s ability to function in the facility by reducing flashbacks, nightmares/sleep, depressive symptoms, 
and hyperarousal. Many youth need to transition from the facility to a group home or home before 
treatment is complete and some have to decide whether to step down to a less restrictive residential 
environment or continue treatment. Efforts to continue treatment via video links are difficult. Group 
homes in the community do not have secure rooms for videoconferencing so youth use administrator 
offices and sessions are often disrupted by people coming in and out of the room. When ongoing 
treatment with the psychologist is not viable or is predetermined to be brief because of an upcoming 
residential transition, the sessions focus on psychoeducation and coaching the youth to advocate for 
themselves with providers who may not have a trauma informed orientation. Interviewees noted that 
youth transfers may occur without consultation with the mental health therapists and may occur at times 
that are disruptive to treatment and result in youth adjustment problems in the new placement. 


Echo Glen also offers gender affirming care and gender evaluations to support youth who are 
investigating gender transition or want to explore their gender identity through partnerships with Seattle 
Children’s Adolescent Medicine and the Seattle Children’s Gender Clinic. Many of the youth seeking 
psychiatric care at Echo also have gender presenting conditions. Treatment follows gender evaluation 
international guidelines and, if deemed appropriate, psychiatric staff (national experts in this field) 
together with adolescent medicine to write a letter to support to the medical staff for beginning gender 
affirming hormone treatment. The process is reviewed by both Echo Glen and JR medical directors. 


Other specialized treatment is offered for substance use disorder and sex offender needs. The mental 
health staff are not directly involved in these programs but take referrals to provide treatment. 


Characteristics of Youth 


The rate of mental health need is very high at Echo Glen. Staff interviewed estimate more than 75% of the 
youth are receiving psychotherapy and/or psychotropic medication. Although needs are as high as youth 
in residential psychiatric settings, youth at JR have generally had less previous mental health treatment 
when compared to the population in child psychiatric settings (e.g., Child Study and Treatment Center). 
The volume of need is a challenge for mental health staff with only one full time psychologist and 
rotating psychiatric trainees who can only take 1 to 2 TF-CBT therapy cases at a time. According to the 
interviews for this report, the lack of sufficient mental health staff leaves most youth waiting for treatment 
or, even when in treatment, insufficient consulting and crisis capacity to manage daily behavior 
challenges. In psychiatric care, psychologists are typically dedicated per living unit at a ratio of about 

10:1 and Echo does not meet this ratio with an ADP of 85 and one full time psychologist. Cottage staff 
burnout was reported as an issue with high turnover and insufficient training for supporting the 
institutional therapeutic model (DBT). Interviewees report that when Dialectical Behavior Therapy is 
happening consistently at the living unit level, one on one treatment can progress more quickly. As noted, 
keeping line staff competent to deliver DBT is a significant challenge. Interviewees for this report were 
consistent in noting problems with current implementation and that more staffing resources would be 
needed to support quality. 


Despite the challenges of meeting needs, interviewees felt that ensuring youth receive adequate treatment 
when transitioning out of Echo to be a similar or an even greater challenge. Providers in community or 
group home settings may be less likely to be trained in trauma and evidence-based interventions for 
anxiety and depression or youth may be in the middle of medication change or trauma work when 
transferred. As noted, telehealth in the JR system is very challenging due to inadequate facilities in group 
homes. Further, the interviewees felt that communication about mental health needs when a youth is 
transferred to a community facility is generally poor. For example, one youth was in the middle of doing a 
cross taper of psychiatric medication when approached about transferring to a group home. The youth 
decided to go to the group home rather than finish the medication tapering with Echo. The Echo 
psychiatrist called the group home to discuss changes in medication but when the youth was taken to a 
local healthcare center, the youth was seen by a mental health professional not licensed to prescribe. The 
provider said the youth was no longer symptomatic and did not refer to a physician/prescriber. The group 
home then followed up with the Echo Glen psychiatrist because the youth had become hypomanic. The 
perception from the Echo Glen mental health staff is that community accessibility to high quality 
treatment is inconsistent. 


For youth being released, interviewees felt that the majority would qualify for WISe but are not referred 
to the program. Wraparound with Intensive Services (WISe) is a care coordination model available to 
youth with serious mental health needs and is administered out of the public mental health system. Youth 
must be referred and assessed in order to qualify for services. One challenge noted in the interviews is 
that some youth have been rejected from WISe eligibility for not having a permanent address at release. 
The challenges in transferring youth between facilities and after release leaves many youth without 
sufficient mental health care for months. 


Room confinement policy 


Echo is currently reviewing their room confinement procedures based on performance based standards 
and from administration. Some staff have concerns about whether youth who are actively unsafe need to 
be out of their rooms given the lack of staffing support for managing these cases. But, overall, 
interviewees felt that Echo’s policies around room confinement are already ahead of other facilities and 
confinement is minimal and infrequent. 


Green Hill 


Green Hill School is a medium/maximum security facility for older, male youth and is located in 
Chehalis, about 30 minutes south of Olympia. The facility is notable for housing a number of youth with 
a history of gang involvement. Green Hill is the only facility to have a contained mental health unit in 
addition to mental health treatment available to all residents. The counseling and treatment options are 
otherwise similar to the other facilities with one onsite mental health counselor, an onsite psychiatrist, an 
onsite mental health coordinator and an intake specialist. 


Information for this report was gathered from in person interviews with the GH Psychology Associate 
(also referred to as the mental health counselor), the Mental Health Coordinator and the staff Psychiatrist. 
The Psychology Associated is a licensed mental health counselor (7 years) and is responsible for 
conducting assessments and providing mental health treatment. The Mental Health Coordinator assists 
with intake assessment for transferred residents, completing the violence risk assessment (SAVY) and 
gender expression assessment (SOGIE). The Intake Specialist conducts these assessments along with the 
GAIN-SS and the Suicide Severity Screen (SSS) as part of the initial client information report for new 
residents (new commitments). 


Assessment 


Mental health assessment occurs within seven days of a youth’s arrival to the facility and includes the 
Beck Depression and Anxiety Inventory (BDI) and the UCLA trauma screen. If a youth scores above the 
clinical threshold, indicating their need for treatment, they are reassessed within 1-3 months to gauge 
ongoing need and progress. The cut off threshold is calculated by the electronic health record system to 
produce a mental health acuity score. The BDI is also readministered within 30 days of a youth’s release 
from the institution. 


Youth are identified as needing a treatment from a variety of sources: mental health assessment, staff 
referral, parents/guardians request, and youth will also self-refer, often to request sleeping medication. In 
contrast to referral, some parents disagree with the need for the mental health treatment. According to the 
interviewees, about five times a year a parent will request that their child stop receiving medication and/or 
mental health intervention. 


Treatment 


Youth in treatment receive routine medication management appointments with the psychiatrist and the 
mental health counselor for ongoing psychotherapy. Nurses keep the mental health counselor and 
psychiatrist apprised if youth refuse medication. Typically, noncompliance is due to the youth feeling too 
tired, feeling that the medication doesn’t work or because the youth sells the medication (this most 
commonly happens with stimulants for ADHD or with Wellbutrin). 


Psychotherapy is eclectic with a focus on cognitive behavioral therapy. If a youth has a severe anxiety 
issue, the mental health counselor may use cue exposure (e.g., for social anxiety). Anxiety often relates to 
the youth’s concerns about what will happen to them after release, their future, or how to manage their 
time while incarcerated. PTSD is a common presenting treatment need. Youth will present with being 
afraid to sleep because of the severity of their nightmares. Less frequently, youth will present with 
psychotic symptoms or self-harming behavior. The frontline treatment in the institution for more severe 
psychiatric needs is medication followed by supportive counseling. For example, in a self-harming case, a 
youth was swallowing glass and switching the youth to Lithium stopped the behavior. 


As noted by the counselor, treatment goals are typically focused on creating pathways for the youth’s 
positive development. Depending on the youth’s length of stay, sessions focus on supporting a youth’s 
successful reentry into the community or the youth’s ability to cope within the facility. Length of 
treatment is depends on the youth’ interest in continuing to engage in treatment and often last the entire 
length of stay. The counselor also worked with a small number of families to support a youth’s treatment, 
particularly as these youth prepared for release. These sessions focus on developing plans for reinforcing 
youth positive behaviors and managing mental health needs. The mental health counselor noted that these 
were helpful but the facility does not have the capacity to do this work for each youth. 


The mental health counselor and psych associate also provides consultation to line case 
management/counselor staff. The line counselor staff are expected to meet with youth routinely to offer 
counseling focused on the use of DBT skills, problem-solving and coping within the facility. Counselors 
may have masters degrees but more often have BAs or AAs and becoming a counselor is considered a 
promotion from a security level position. Counselors run unit level DBT groups. 


Youth with acute mental health needs and lower social functioning may be placed in the specialized 
mental health unit (Baker). Oftentimes, the determining factor for whether a youth is placed is concern for 
their safety in the general population. Youth with mental health needs that have more aggressive 
behaviors may be placed in a different mental health unit (Cypress). The mental health units have more 
scheduled group treatment time and the line staff/case managers are generally more experienced and 
motivated to be delivering the Dialectal Behavioral Therapy (DBT) components than in other units 
according to interviewees. 


Transitions 


Transition support from the facility to community focuses on enrolling youth in Medicaid and 
Wraparound with Intensive Services (WISE). The mental health coordinator works with the Health Care 
Authority to ensure the youth is Medicaid enrolled. Typically, HCA will tell the youth what health plan 
they are on rather than engaging with the youth to have a discussion about the best healthcare option for 
their needs. The Green Hill youth committee has a goal to support better decision making around health 
plan enrollment but the effort is not well coordinated with the mental health coordinator and HCA 
enrollment process. Currently, SeaMar staff meet or have a brief phone call with the youth to confirm the 
youth’s address and then will assign the youth to a plan. 


A youth is given their Medicaid number (Provider One) in their personal belongings at release. The 
Healthcare Authority is expected to unsuspend the youth’s coverage on their release date in order to 
trigger the release of the youth’s insurance card in the mail. The staff at Green Hill have no way to tell 
whether this is routinely happening and only hear about cases where it has not happened (for example, if 


the HCA staff responsible for unsuspending coverage was on vacation). It was not clear to the mental 
health staff interviewed how consistently parole ensures that youth are enrolled in Medicaid or engaging 
in WISE teams. Oftentimes youth do not have phones and are only in contact with JR after release when 
they attend parole meetings. Conversations about a youth’s interest in engaging in WISE occurs 60-90 
days prior to release. If a youth is interested, Green Hill mental health staff work with the regional mental 
health coordinators to identify a local provider. The local provider conducts the eligibility screen (CANS 
assessment) with the youth. Only a handful of CANS screens have been performed for Green Hill youth. 
The impression from Green Hill staff is that youth do not typically follow up with local WISE services. 


Characteristics of youth and treatment 


Staff interviewed noted that mental health counseling is helpful when delivered but capacity is not 
sufficient to address the volume of mental health need that line counselors are not consistently 
implementing the expected DBT skills. Further, transitions in care after release are difficult due to the 
need to rely on HCA and community providers for follow up. There is concern about the appropriateness 
of using line counselors to manage youth privileges and consequences as well as deliver effective 
counseling. The impression from the interviewees was that many counselors are not interested or do not 
feel capable to deliver DBT but because being a counselor is a promotion over security level staff, many 
pursue the additional certification. There was also concern that DBT may not be the best fit for a number 
of the general population youth who may need less specialized support and more focus on general 
problem-solving and coping skills. 


Room confinement 


According to interviewees, room confinement at Green Hill is expected to be minimal. Fifteen minute 
“time outs” are used fairly frequently for youth who do not stop behaviors after being approached. Green 
Hill has a quiet room, a cell with a camera that may be used when youth behaviors are particularly 
aggressive. 


Naselle Youth Camp 


Naselle Youth Camp is a medium security facility for males aged 16-21. The camp is located in 
the southwest area of Washington State approximately 2.5 hours from Olympia, WA. It is 
considered a general facility not equipped to manage acute mental health needs. Youth at Naselle 
have the opportunity the earn high school and general equivalency diplomas (GED), forestry 
work training in collaboration with the Department of Natural Resources or aquaculture training 
in collaboration with the Department of Wildlife. Youth generally stay in the facility less than 
one year. 


The interviews informing this summary were conducted over the phone for two hours with the 
Associate Superintendent and the Mental Health Coordinator on 11/19/2019 and with the 
Psychology Associate, licensed MSW on 11/23/2019. The facility’s licensed Psychologist 
resigned and Naselle is currently soliciting applications for her replacement. The geographical 
isolation of the camp is attributing to low recruitment success. Other staff involved in mental 
health oversight or treatment not interviewed include the general physician for Naselle and 
Juvenile Rehabilitation medical director; the facility RN who participates in mental health triage 
planning; and a contracted ANRP who provides psychiatric consultation and prescribes 
medication. 


Assessment of mental health needs 


Naselle staff first become aware a youth has or may have a mental health need through the intake 
report provided by JR diagnostic coordinators stationed in regions throughout the state. 
Diagnostic coordinators conduct assessments prior to the youth arriving at a JR facility, most 
often these are conducted while the youth is still housed in their local juvenile detention center. 
The assessment contains quantitative items that flag for mental health need including whether the 
youth is currently or was previously prescribed psychiatric medication, previous hospitalization 
for a mental health need; and is currently or was previously receiving mental health counseling. 
In addition, the process accounts for presenting or history of self-harm or suicide ideation, or 
behaviors, as well a current endorsement of psychosis. Naselle may also get information directly 
from the detention center, particularly if the youth was disruptive and had a difficult time 
behaviorally adjusting to their detention stay. 


When a youth arrives at Naselle, they undergo further screening for suicide and self-harm 
(Suicide Self Harm Scale, SSS), behavioral health needs (GAIN-SS), trauma history (UCLA 
trauma screen), depression and anxiety (Beck Depression Inventory, BDI), a medical screen for 
physical detoxing symptoms, educational assessment, substance use assessment, records review 
and requests for further information from families, home schools or community-based sources as 
needed. 


Treatment planning 


Treatment planning is incorporated into Mental Health Triage meetings and reentry planning 
meetings (RTM) beginning within the first two weeks of arrival (pre-RTM meeting), the 
standard 28 day RTM meeting, and final RTM prior to release. Indications that a youth may have 
an active mental health need from any of these screens and informant interviews results in a 
youth’s placement on the “Active Psych Referral List” and a discussion of mental health needs at 
the pre-RTM meeting. This list is an internal tracking document that identifies a youth in need 
of additional mental health screening/assessment and/or consultation, and prompts a meeting 
with the psych associate during which the youth can discuss symptoms they’re experiencing and 
any history of interventions. Youth who potentially have an indicated treatment need beyond 
behavioral-based interventions and supported by objective information, are brought to a Mental 
Health Triage meeting involving the multidisciplinary healthcare team, which may consist of the 
mental health coordinator, the psych associate, the contracted ARNP, the onsite RN and the JR 
medical director (also the facility’s Medical Doctor). 


The Mental Health Triage team discusses the youth preferences for treatment, often related to 
medications they are currently on, would like to get off or would like to start. A common youth 
request for medication is to assist with sleep and may identify a specific medication as part of the 
request. If a youth requests sleep medication, the team will recommend a sleep study. For all 
requests for mental health treatment, the Mental Health Triage team discusses whether to start 
with behavioral protocols or medication; occasionally, behavioral interventions have been 
exercised without noticeable improvement, resulting in discussion about medication treatment. 
More recently, Naselle has received a higher proportion of youth with acute mental health needs 
than past years as will be described in more detail below. 


Treatment 


Naselle provides three levels of treatment. For youth prescribed psychiatric medications, the 
contracted ARNP delivers consultation to clients up to three times a month. Delivery of service 
occurs in-person and through use of Telemedicine (video conferencing). When Telemedicine is 
used, youth meet in a private area in the facility over a secure video connection. 


The psych associate provides brief (typically 5 session) psychotherapy for youth indicating a 
willingness to work on a particular issue (e.g, trauma). 


The facility also provides a trauma group, facilitated by the psycho associate, that blends 
components of T-4, Cognitive Behavioral Intervention for Trauma in Schools (CBITS) & 
Adverse Childhood Experiences (ACE) materials into the overall group cycle. Residents emerge 
better able to identify trauma responses & are encouraged to pair these with evidenced-based 
interventions. In previous years, Naselle offered SafeDates, an interpersonal violence prevention 
program. 


All youth, regardless of mental health need, are assigned a case manager who works with the 
youth on goals related to emotional regulation skills throughout their time at Naselle using the 
Integrated Treatment Model and Dialectical Behavior Therapy (DBT) principles. This includes 


using behavior chains to help youth understand what motivates their behavior and coaching 
youth on various emotional and social skills. Naselle staff reported that the use of the ITM/DBT 
skills varies widely by staff member, largely due to gaps in training and tenure. 


Characteristics of youth at Naselle 

Naselle is not considered an acute mental health facility. However, staff reported that a high 
percentage of youth are coming into the facility with previous or active substance misuse 
(>80%), depression or anxiety. Naselle is not equipped to receive youth who have acute, 
untreated psychosis or are actively suicidal; initial placement for these youth would typically be 
either Green Hill or Echo Glen, where additional resources are available to address these issues. 
However, some youth do not manifest these needs or symptoms until after they are placed at 
Naselle and no formal mechanisms are in place to have these youth transferred to a different site. 
Currently, JR diagnostic coordinators conduct the intake assessment and the algorithm 
programmed into the computer system recommends a placement. The coordinator is able to 
intervene with central JR if they feel the assessment recommendation is not the best fit. 
However, if youth are transferred to Naselle and subsequently exhibit or report significant mental 
health needs, there is no process in place to review the initial placement decision and consider an 
alternative placement. 


The staff believe that some youth who would historically be placed at Green Hill for mental 
health needs are being sent to Naselle, and the staff anticipates that this will become more 
common as facilities transition to serving youth up to age 25. Naselle staff do not currently have 
training in how to effectively interact with youth with severe cognitive delays, autism spectrum 
disorders or comorbid diagnoses (e.g., substance misuse/dependence, mental health and 
cognitive/neurodiversity needs). They are putting more youth on suicide precautionary level 
than in the recent past (>1 year ago). Naselle staff feel that the staff to youth ratio is too high for 
adequately treating youth with mental health needs in a milieu setting (smallest unit is 20 youth). 
This results in a lack of integration between a youth’s mental health treatment goals and how the 
youth is managed within their unit. The emphasis of the facility is safety, psychiatric medication 
management, communicating on a youth’s progress during team meetings and communicating 
information to community or group home providers prior to transition. Case managers assigned 
to work with youth on their general behavioral goals in the milieu have received limited training 
on how to evaluate how mental health needs are intersecting with self-harm and aggression. 


Room Confinement 


Youth are confined to their room for physical aggression and defiance towards staff. 
Youth stay in their rooms until they demonstrate emotional stability and make a verbal 
commitment follow program expectations te and act safely. A high number of cases (80% 
estimate) are out of confinement status as soon as they can verbalize a commitment and no 
longer present as a safety risk to the milieu. Some kids may have been in a bad fight and go in at 
6pm that night and not out until 10am next morning; this accounts for a minimal percentage of 
all confinements. Youth are out of rooms programmatically more than they ever have been (vs. 
in room programming). Residents may program in their rooms during brief (i.e. 30 minutes) 
periods to accommodate te staff shift changes and meetings, etc. Incident reports reflect staffs 


efforts to coach and re-direct youth multiple times before send the youth to their room for a time 
out (up to 15 minutes) or longer (confinement status). Fights, assaults, destruction of property 
and, repeated failure to follow staff direction or program expectations are common incidents 
leading to placement on confinement status. While on room confinement youth are visually 
checked no less frequently than every 15 minutes at staggered (unpredictable) intervals. Weekly 
administrative, management and living unit team meetings review the use of room confinement 
while generating ideas and forming alternative interventions to reduce/replace use of 
confinement Interviewees viewed this policy as a work in progress with some concerns about the 
underlying assumptions, e.g., kid must be dysregulated to act out. With some kids willful, 
intentional behavior is not necessarily a product of dysregulation and the facility needs support 
developing and implementing alternative behavior modification plans if room confinement is not 
available. 
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Executive Summary 

Washington state Juvenile Rehabilitation (JR) has long been on the forefront of developing and 
implementing evidence-based behavioral practices for the care, management, and treatment of the 
youth it serves. This includes its inclusion of a comprehensive approach to assessing, managing, and 
treating youth who are at risk for death by suicide and/or engage in suicidal and/or non-suicidal self- 
harm behaviors. The overarching approach to suicide and self-harm prevention was developed in 
collaboration with national consultant, Lindsay M. Hayes, PhD in 2001. Prevalence and risk factor data 
have been subsequently updated by JR personnel. The Suicide and Self-Harm Screen (SSS) and protocol 
was developed in partnership with Andre lvanoff, PhD from Columbia University. Dr. lvanoff is a DBT 
expert who has devoted her career to helping disseminate DBT to complex, multi-diagnostic suicidal and 
self-harming persons. The intention of this review was to determine the extent to which the suicide and 
self-harm Policy 3.30 and associated procedures are fulfilled at JR and to make recommendations as 
needed. To make this determination, several JR site visits were made during November, 2019; 
subsequent telephone interviews were conducted in March, 2020. Whenever possible, conversations 
included a range of staff (leadership and frontline youth counselors), visiting the units, and speaking 
with youth. 


Key Findings: 


- The Policy 3.30 that governs the procedural practice of assessing, managing, and treating 
suicidal and self-harming youth is comprehensive, sophisticated, and contains numerous 
safeguards to ensure each at-risk youth’s situation is carefully assessed. Counselors conduct a 
comprehensive interview guided by a desk manual that includes well-formed scripts to help 
ensure that the questions are sensitively asked and that the youth is oriented to the direction of 
the interview. Staff are also required to document their own observations as well as relevant 
historical information and to determine the Safety Plan Level. All of this occurs in the context of 
an online electronic environment that ensures completion of each of the questions. The 
document is then electronically submitted to a Designated Suicide Prevention Specialist (DSPS), 
a person appointed by the medical director who has more elaborate training in suicide 
prevention. The DSPS can either approve or deny (override) the counselor’s recommendation. 
The SSS is a dynamic tool that is used whenever there is a change detected in the youth’s 
behavior/risk. 


- All staff working with youth are required to complete multiple trainings on suicide and self-harm 
prevention. This includes an initial facility-based training within the first week of their 
employment to ensure that they understand the policies and procedures. Within the first 
several months of their employment, they are required to complete a comprehensive full-day 
(eight hour) training in suicide prevention. This training is very thorough, comprehensive, and is 
likely more than most licensed clinicians receive in graduate school. Brilliantly, employees are 
expected to study and memorize risk and protective factors before arriving at their training. In 
addition, they are expected to locate their Emergency Suicide Response Kit in their facility and 
observe an intake SSS. Each of the required assignments includes a worksheet the new 
employee uses to guide their experience. These training approaches are sophisticated and 
rigorous. 
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The SSS is not really a screening tool, but is instead a fairly lengthy comprehensive assessment. 
All staff that | spoke with found the tool easy to administer and noted that it is reliably 
conducted in accordance with Policy 3.30. The DSPSs | spoke with found that it provided them 
with the information they needed to validate or reject the Suicide Prevention Level assigned by 
the counselor completing the interview. The majority | spoke with thought that the SSS 
contained redundancy and might be consolidated to reduce time to administer. On average, a 
typical youth might complete the SSS on four occasions, yet there is only one version that 
contains items that are unlikely to change. (Please see Appendix B, Observations for the Suicide 
and Self-Harm Screening). Additionally, the second part of the SSS contains redundancy with the 
first part. 


The vast majority highlighted how staff shortages and low staff retention significantly reduce the 
overall clinical quality of the SSS. A number of factors account for this fact: First, most newer 
staff are simply not sufficiently trained to conduct more than a simple, rudimentary SSS and 
treatment plan based on findings from the SSS. In this instance, the problem is one of lacking 
the capability because they are new and simply do not have the lived experience as a more 
veteran staff has. Second, even when a counselor has the capability, they simply do not have the 
time to do more than a cursory barebones assessment and treatment plan. This may be 
particularly true on units with high degrees of aggressive, assaultive behavior (more than the 
norm within JR) and where staff are “constantly putting out fires”. 


The policy does not allow for exceptions or for a DSPS to override the policy in certain 
circumstances which in some instances may have an iatrogenic effect. Concretely, there are 
youth who will use suicidal threats to avoid something they do not want to do, to get staff 
attention and time, and/or to cause disruption on the units when staff are already overwhelmed 
with staff shortages. When it is known that the behavior is solely for these reasons, following 
the policy simply reinforces/strengthens the very behavior the policy seeks to reduce. Those | 
spoke with who serve as DSPSs were mixed about how to solve this problem: while they 
universally preferred to have the capacity to deviate from the normative policy, unless it were 
codified in the policy and contained safeguards to guard against drift, they would be hesitant to 
change the policy. 


The treatment plan to address suicidal and non-suicidal self-injurious behavior requires 
sophisticated behavioral procedures that are contained within Dialectical Behavior Therapy 
(DBT). These include: behavioral chain analysis, solution analysis, commitment strategies, and 
strategic use of the diary card. Even under optimal circumstances, these therapeutic strategies 
are complex and difficult to do well without extensive training, supervision, and consultation 
that includes monitoring the counselor’s actual clinical capacity. (See DBT report). However, 
Washington State JR, despite a golden-years decade of learning and implementing DBT and 
other evidence-based behavioral therapies in the late 1990s, is not currently able to offer 
optimal circumstances. Staff shortages and staff turn-over have significantly compromised JR’s 
ability to train up its workforce in these and other procedures and to maintain whatever training 
gains it makes. Non-competitive wages make it particularly challenging to recruit personnel at 
all levels (AA, BA, MA, PhD). The situation is even more dire in rural areas. Without staff 
retention and comprehensive training, consultation, and supervision, it is difficult how 
counselors can be expected to actually implement an effective treatment plan that reduces 
suicide and self-harm risks. Blocking procedures and the SPL procedures will ensure that those 
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at risk in the moment do not have access to lethal means and methods to cause self-injury or 
death by suicide. This is different however than having a carefully developed treatment plan as 
specified in Policy 3.30 that will serve them well after they leave JR. 


Main Recommendations 


1. 


Reduce Staff Shortage, Improve Retention. For all organizations, the capacity to implement its 
vision hinges on its employees learning and doing the requisite tasks required to fulfill the 
vision. Staff shortages make it difficult for staff to attend trainings and have the required time to 
do what is required well. Additionally, staff turn-over results in a steady stream of beginners 
without building system competence. 


Reduce SSS interview process. There are a number of ways that the current SSS could be 
retooled to reduce redundancy, increase collaboration, and improve efficiency while remaining 
a gold-standard tool. Several suggestions are offered in the Appendix B. It is recommended that 
JR consider convening a task force comprised of all relevant stakeholders (including youth) and 
outside suicide experts to streamline the method. To ensure its integrity and excellence, the 
final version could be reviewed by a panel of adolescent/young adult suicide experts. 


Ensure Adequacy of Training for DBT Core Competencies. While the SSS Desk Guide provides 
scripts to read to conduct a thorough suicide/self-harm risk assessment, no such script is 
feasible for developing a youth-specific treatment plan let alone conduct the quality of 
treatment required by Policy 3.30. The only way a counselor will be able to actually treat that 
which he/she/they assessed using the SSS is by receiving comprehensive training in the 
treatment procedures, including ongoing consultation and supervision. Treating complex, multi- 
diagnostic youth (particularly those who do not want to engage in treatment) is not simple. The 
training is extensive, comprehensive, and ongoing. Without adequate training, supervision, and 
consultation that transfers to developing and executing a patient-specific treatment plan for 
treating suicidal/self-harm behavior, there is little reason to believe that we will reduce the 
youth’s long-term suicide risk. 


Carefully Consider Providing DSPS Option to Deviate from Standard Procedure. The problem of 
youth stating they are suicidal in order to avoid something they do not want to do or get 
something they want is not an uncommon problem in juvenile justice facilities. Having a senior 
member of leadership have the capacity to override the policy when it is abundantly clear 
suicidal behavior is solely an operant behavior is wise. It is also wise that change to the policies 
and procedures are done mindfully and do not create another set of problems when solving 
one. One solution may be to create a special treatment plan for these youth that is reviewed 
and approved by two or three other DSPSs well in advance of the time that it will be needed — so 
that no such decisions are being made in the moment to deviate from the policy. This approach 
might also require two DSPSs to sign off on the permission to deviate in the moment. 


Consider Having a Designated Suicide Expert Available for more Complex Cases and to 
Conduct Quality Assurance Review of SSS. Given staff shortages and problematic staff 
retention, JR could benefit from having a QA process to cross check whether problems assessed 
in the SSS and behavioral chain analysis are being effectively treated using the most effective 
strategies. 
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ASSESSMENT 


Materials Review. In advance of visits and interviews at facilities, the JR Clinical Director provided 
numerous documents for my review. Each document is reviewed in Appendix B. When relevant, detailed 
observations for each are noted. 


OBSERVATION AND INTERVIEWS 


Leadership staff and counselors were interviewed at Green Hill School (November 8, 2019 & November 
12, 2019), Naselle Youth Camp (November 25, 2019), Touchstone (November 27, 2019), and Echo Glen 
Children’s Center (November 27, 2019). Observations are detailed below.‘ Individual and group 
interviews with seven staff, the majority of whom were DSPS providers, were conducted during the 
week of March 9, 2020. 


1 These observations focus specifically on those relevant to my assessment of the suicide/self-harm policies and procedures that have not already been 
detailed in the DBT report prepared by me for Washington State JR. Several observations noted in the DBT report are relevant to the quality suicide/self- 
harm assessment, management, and treatment. These include: the significant problem of staff shortage and high rates of staff turn-over that 
compromise JR’s ability to have a workforce adequately trained to fulfill its clinical mandate and vision. | note in the DBT report that these factors result 
in uneven delivery of evidence-based behavioral interventions because only a few staff have the requisite core competencies to do the treatments. 


Portland DBT 
ore INSTITUTE 


APPENDIX A: QUESTION INFORMING REPORT 


Washington State JR provided the following questions for consideration during my visits and to guide 
the summary of my findings. 


1. Are youth being appropriately matched with treatment based on need? 
a. What assessment is being used for program eligibility? 
b. Is the assessment being used appropriately? 
Policy 3.30 and its associated practice procedures provides a comprehensive framework for screening, 
managing, and treating suicidal and non-suicidal self-harm behaviors throughout all parts of the system. The 
primary tool used by Washington State JR (the Suicide and Self-Harm Screen; SSS) provides for three main 
levels of observation (SPL1, SPL2, and Observation Level) for those deemed at risk by virtual of a number of 
specific, observable and historic information. The level of intervention and oversight of youth is contingent on 
their level. The assessment is being used appropriately though some enhancements to the SSS process could 
help ensure that it is easier to administer -- and potentially more reliably administered. Please see 
recommendations. 


2. Is the treatment high quality? 
Staff shortages and staff turn-over, combined with limited comprehensive training and supervision, 
compromise the routine application of high-quality treatment. Many of the veteran staff have retired or 
moved on. Those that do have the capability do not often have the time. The quality of care a youth receives 
is highly variable — from a brand new clinician who provides garden variety therapy in response to the SSS 
(albeit with good intention) to some of the most talented clinicians in the country. 


b. Are youth receiving the appropriate dosage? Unlikely do to staff shortages, poor staff retention, and 
limited training. Also, in some cases, the appropriate treatment is no intervention in cases where it’s fully 
operant behavior. 

c. Do staff receive the correct type and amount of training? In comparison to DBT training, it does seem that 
immediate training in assessing suicidal and self-harm behaviors occurs routinely. The quality of training is 
outstanding. 

d. Does there seem to be variation by location? (between institutions, between institutions and community 
facilities) In units that have higher than usual levels of aggressive behavior and are larger, it is likely that the 
quality and availability of treatment varies. 

Please see attached report. 


3. How well is the treatment integrated with the assessments, other treatments, and reentry 
planning? While the policy and practice procedures require thoughtful and ongoing screening, assessment, 
and management of suicidal and non-suicidal self-harm behaviors across the continuum, the ability of a 
counselor to do the requisite steps is contingent on receiving training, consultation, supervision, and ongoing 
support doing DBT. Currently, for example, the treatment of suicidal and self-harm behaviors is based on a 
counselor’s capacity to: conduct a behavioral chain analysis, develop safety and stability strategies to use in 
the face of a crisis (solution analysis), gain a commitment to use skills rather than engage in suicidal and/or 
self-harm behaviors, and track/target it all on a diary card. Each of these tasks, including creating a 
treatment plan that addresses short term and long term risk, is complex and difficult to master, even for PhD 
level behavioral clinicians. It is hard to imagine given the extensive staff shortages and turn-over that the 
majority of counselors have the requisite skills to actually DO what’s called for to fulfill Policy 3.30. 


4. Does the current quality assurance plan adequately measure the treatment model? /t does. 


5. Do you see any evidence that would concern you that treatment access varies by race, ethnicity, 
age, or gender? No evidence. 
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What recommendations do you have to improve implementation given the current level of 
resources? Please see recommendations. 


What additional resources would you recommend to improve treatment quality and integration? 
Please see recommendations and observations. The most significant suggestion is to streamline the SSS 
process. While called a screening, it is a fairly comprehensive interview that includes redundancy (asking 
repeatedly about distal/historic factors already known to JR). 


JR will be serving youth up to age 25 in the future. What advice would you give the agency as 
they prepare to provide this treatment for older youth? To incorporate risk factors, protective factors 
and other risk and protective factors for older youth. 
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APPENDIX B: MATERIALS REVIEWED 


Poticy 3.30 & MEMO oF 9/27/2019 ENTITLED, “CHANGES TO SUICIDE PRECAUTION LEVEL (SPL) AND SUICIDE AND 
SELF-HARM SCREENING (SSS) POLICY AND PRACTICE. The memo highlights changes to policy and practice 
based on feedback from auditors with the National Commission on Corrections Healthcare. An updated 
policy and practice for the Suicide Precaution Levels (SPL) and SSS was to be released on November 7, 
2018: SPL 1-3 combine into SPL1 and SPL2, and SPL4 changes to Observation Level. Each level is 
described in the memo. 


Several notable (to this reviewer) observations about the policy itself: First, the policy is comprehensive 
and contains best practices in the prevention and treatment of self-harm and suicidal behaviors. 
Elements focusing on suicide include: a clear expectation that all staff who work with youth are oriented 
to and trained in suicide/self-harm screening, intervention, and policy; training, testing, and supervision 
will be ongoing to ensure excellence of care; each facility has a quality improvement process to ensure 
the policies and procedures are being followed at all levels; each facility will have Designated Suicide 
Prevention Specialists (DSPS) who are appointed for their own internal expertise in suicide prevention 
precautions, screening, and treatment; that a designated response team is at the ready in the tragic 
event of a suicide. Elements focusing on non-suicidal self-harm behavior include detailed action for how 
to assess, track, and treat. These include: assessing self-harm behavior when it occurs using a behavioral 
chain analysis; identifying safety strategies to use for emotion regulation instead of engaging in self 
harm; getting commitment from the youth to not engage in self-harm; develop and update treatment 
plan that targets and treats self-harm; use and review of diary card. 


SUICIDE AND SELF-HARM SCREEN (SSS; BLANK FORM). | reviewed a blank version (six-page document) of the 
SSS. This is a comprehensive form that includes a thorough assessment of current and proximal factors 
of risk for suicide and self-harm. The form includes a 21-item assessment with youth about current and 
historic factors (e.g., Currently using prescription medications? Have you been more anxious than usual? 
Have you ever had thoughts or urges to harm or kill yourself?). A Staff Observations/History 17-item 
assessment for completion by staff based on the known history and observations. Some items are 
repetitive, but consolidated (e.g., Has youth expressed or demonstrated: sadness, irritability, 
anxiousness, hopelessness, or despair?) as well as observational in nature: has youth demonstrated 
withdrawal? Upon completing the initial screening items, staff are required to select a recommended 
Suicide Precaution Level (SPL): Level |, Level II, Level Ill, Observational Level, or No Level. Consistent with 
JR policy, the SSS includes information about whether the parent was notified, the date of notification, 
and additional comments. Finally, SSS includes the approval status (Approved, Denied) by the 
Designated Suicide Prevention Specialist (DSPS). 


Observations: 

1. With a total of 21-items to ask/review with the youth and 17-items for later completion by the 
staff, this tool is more of an assessment device than it is a “screen”. It is length, repetitive 
(asking youth, then completing similar items in a section section), and requires additional 
comments (staff time) if an item is positively affirmed. While the intent of this assessment is to 
ensure all bases are thoroughly covered at all moments to reduce suicide and self-harm from 
occurring and ensure best care, it seems unnecessarily complex and redundant particularly in an 
already burdened system with significant staff shortages. Within healthcare contexts, screenings 
tend to be brief (a handful of items). If a person has screened positively, then a more 
comprehensive questionnaire is provided followed by a still more comprehensive assessment. 
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For example, National Institute of Mental Health researcher, Lisa Horowitz, PhD, MPH and her 
colleagues have recently published the Ask Suicide-Screening Questions (ASQ) screening method 
containing four screening items followed by a fifth (“Are you having thoughts of killing yourself 
right now?”). lf yes, then the clinician moves to the next level of screening. 


2. The same tool is used for historic/distal information (e.g., “Has anyone in your family or close to 
you harmed or killed themselves?” , “Do you tend to do things impulsively or before think about 
them?” as for proximal/immediate information (e.g., “Have you been sadder than usual?”, 
“Have you had something bad or stressful happen to you lately?”). lt seems as if the same SSS is 
used for the initial assessment and throughout the youth’s involvement as JR — thus making the 
items very redundant. 


3. The Staff Observation/History items consolidate similar items and may provide a helpful method 
for streamlining questions for youth: Has youth expressed or demonstrated: sadness, irritability, 
anxiousness, hopelessness, or despair? |f affirmed, then details are required. A similar approach 
could be taken to reduce the sheer volume of questions. 


4. Some approaches for suicide assessment, like the Collaborative Assessment and Management of 
Suicidality (CAMS) approach are done in a fashion that helps build and establish rapport. The 
CAMS Suicide Status Interview (SSI) provides the structure to ensure completion of the task. 
Because the method is highly collaborative, it also engenders a building of rapport and creating 
of therapeutic alliance. In other words, the process itself is therapeutic. The SSS feels more like a 
research tool to gather information in an impartial, standardized, research-validated manner. 
Other documents strongly encourage users to not deviate from the specific questions as they 
are because the method is validated. The CAMS is just one method of how a suicide risk 
assessment can be delivered in a reliably and in a method that engenders compassion and 
discussion. Such an approach then might allow for only one section — with additional notes 
provided by counselor if they and the youth are in disagreement and/or if the counselor wish to 
make observations that they prefer to not discuss openly with the youth. (Therapeutically, even 
discussion of what they observe seems both clinically relevant and helpful). 


STUDY GUIDE FOR PREPARATION FOR INITIAL 8-HOUR FULL DAY SUICIDE PREVENTION TRAINING. This nine-page 
document provides a high-level overview as well as tracking of content they are expected to do as 
preparation for the day-long training. A checklist of required preparation includes: watching “Teen 
Suicide” video; being prepared to recite suicide specific information about warning signs, risk factors, 
high risk periods, protective factors from memory; review of Policy 3.30, the SSS Desk Manual 2019, and 
Supervisor Conditions; locate and examine the Suicide Response Kit; and observe an Intake SSS. Staff 
and their supervisor are required to sign off on having completed these preparation tasks. To aid the 
trainee in actively engaging in the observation of the intake, a SSS Observation Sheet helps guide their 
awareness and learning of relevant domains and content. A Warning Signs, Risk Factors, High Risk 
Periods and Protective Factors document summarizes content to help learn the relevant content. The 
Suicide and Self-Harm Observation and Documentation Grid details the Level Placement Criteria, 
Observation, Documentation, Supervision Conditions, Reduction Timeframes, Required Mental Health 
Consultation for each of the four levels (SPL1, SPL2, SPL3, and Observation). SPL Supervision Conditions 
are summarized for each level, including standard. An Emergency Suicide Response Kit worksheet 
directs their effort to locating where the kit is located on their unit and to the supplies that should be 
contained in the kit. Finally, the trainee indicates when (date, time, location) they are scheduled to 
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attend the 8-hour Full Day Suicide Prevention Training. 


Observations: The overarching approach taken here by JR is very clever and sophisticated from a 
training perspective. Specifically, staff are required to come to the full-day already prepared, where the 
basics are already known to them (e.g., they’ve memorized risk factors, read the policy, inspected the 
Emergency Suicide Response Kit). Presumably, this allows for a more thorough discussion of content, 
role play, and more thoughtful questions. Second, both staff and their supervisor are required to sign 
that they have actually done the task. Having the supervisor sign off is particularly smart — to ensure that 
in fact the tasks have been done and barriers to completion are known, solved. Finally, inclusion of the 
worksheets to direct their focus for what to observe during the initial SSS and their inspection of the 
Emergency Suicide Response Kit moves what could be a non-focused learning exercise to one that is 
focused on the relevant training objectives. 


SUICIDE AND SELF-HARM DESK MANUAL. This 21-page document, last revised January 4, 2019, provides a 
brief summary of the JRA Policy 3.3.0 then detailed instructions for when/how to complete the SSS. 
Important reminders include: conduct the SSS in confidential, distraction-free environment with a goal 
of having the youth be a willing and truthful partner during the interview. Screen shots of the online 
form are included as instruction. To ensure effective administration of the SSS interview, 
instructions/suggestions for what to say are included as well as a therapeutic flow to the interview that 
includes thorough orientation to the process itself. Bolded text in a section entitled, “Structure of Youth 
Interview”, users are informed: “The questions should first be asked as written. If the youth does not 
understand, even after the question is repeated, reword the question...This is important because the SSS 
has been validated by Columbia University. Deviating from the original question could invalidate the 
tool.” 


The flow of the SSS interview guides the user through the steps and helps frame the transitions from 
one section to the next: INTERVIEWER: Now | want to ask you about self-harm and suicide. These 
questions are personal, but important. Some of these questions ask about ever in your life and some are 
about right now. Please ask me to clarify if you don’t understand.” Scripts for follow up questions and to 
validate the pain and difficulty of their experience are also included: INTERVIEWER: If the youth answers 
yes, then say, “That must be hard for you”, or make other appropriate empathetic response. 


Observations: 


1. The desk manual is well-organized, easy-to-use, and naturally follows the flow of the SSS 
interview. The Youth Self-Report interview scripts are clinically thoughtful and provide a novice 
clinician with the capacity to sound like a master clinician, as the youth is oriented to each 
section of questions with a sense of what to expect and empathic responses are provided. 


2. There are no scripts provided for the discussion with the youth’s parent(s)/legal guardian, nor is 
there principle-based guidance for these calls beyond what is required by Policy 3.30.23.8 and 
Policy 1.40.16. 


3. The recommended script includes use of “self-harm” and “suicide” - words that may be scary 
and off-putting to youth: INTERVIEWER: Now | am going to ask you about self-harm and suicide. 
These questions are very personal but important to ask. The SSS interview itself does not use 
this explicit language. Many times the questions are framed as they are in the SSS in more 
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descriptive (less categorical) terms: “In the past week, have you wished you were dead?” “Have 
you ever tried to kill yourself”, etc. The script might be revised to something more like: 
INTERVIEWER: Now | am going to ask you about times in your life (or past week or now) when 
you wanted to or did harm yourself or wanted to end your life. 


SSS, SPL, & Supervision Condition Changes in ACT: Quick Info Sheet. This three-page document 
highlights the changes made to the SSS in response to an audit by the National Commission on 
Correctional Health Care. The specific focus is on when and how the necessary approvals are 
strengthened. Three possible outcomes for review (by supervisor) of the SSS are noted: Approved, 
Denied, and Superseded. This latter outcome allows a supervisor to close the loop on previously 
submitted SSS that has since been revised/updated based on more current information. 


SSS/DSPS QA Form (June 3, 2019). This one-page six-item document is intended as a quality assurance 
check to ensure the policy and procedure is implemented routinely and accurately. 


Revised Prevention, Assessment and Understanding of Suicide (PAUSE) Trainer’s Manual: A curriculum 
developed for use in residential juvenile rehabilitation facilities in Washington State, November, 2018. 
This 290-page document details the training curriculum for all aspects of the suicide prevention training 
plan (in compliance with Policy 3.30). The curriculum was developed initially in 2001 with assistance 
from national consultant Lindsay M. Hayes, PhD and has since been revised overtime by John T. Bolla, 
MS, CDP to stay current with current statistics and research. A number of the documents included in this 
curriculum (worksheets in preparation for 8-hour training) are included. The curriculum is 
comprehensive, highlights key points for trainer and provides very specific talking points to ensure 
training is standardized across training. True/False tests and myth/fact quizzes are included. 
Demographic data on JR Youth Suicides from 1990 — 2017 are reviewed. 
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APPENDIX C: QUALIFICATIONS OF REVIEWER 


Linda A. Dimeff, PhD, is Chief Scientific Officer at the Evidence-Based Practice Institute, Inc; Institute 
Director at Portland DBT Institute, Inc., and Clinical Faculty in the Department of Psychology at the 
University of Washington. Since 1994, Dr. Dimeff has collaborated closely with Marsha M. Linehan to 
develop and evaluate an adaptation of DBT for substance-dependent individuals with borderline 
personality disorder; to produce DBT training materials for clinicians; and to train, consult, and supervise 
clinicians in their practice of DBT. She has worked with public and private sector systems throughout the 
world in their efforts to implement DBT. Dr. Dimeff is a recipient of the Cindy J. Sanderson Outstanding 
Educator Award from the International Society for the Improvement and Teaching of DBT. Linda has 
received over 20 federal grants to facilitate the dissemination of evidence-based therapies and has 
published over 55 peer-reviewed publications. She is also the first author of Brief Alcohol Screening and 
Intervention College Students. 


